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Abstract 

A  group  support  system  (GSS)  uses  a  combination  of  networked  personal 
computers,  software  that  collects,  manipulates,  and  aggregates  member’s  individual 
input,  and  human  facilitation  to  improve  the  group  decision-making  process.  Group 
support  systems  are  being  used  in  the  Air  Force  today  in  a  variety  of  capacities  and  in 
particular  by  the  Wamer-Robins  Air  Logistics  Center  (WR-ALC)  to  assess  acquisition 
risks.  GSS  facilitators  at  WR-ALC  are  interested  in  achieving  the  optimal  productivity 
out  of  their  GSS  system.  Prior  GSS  research  has  found  that  certain  structural  and  social 
dimensions  of  GSS  designs  might  influence  whether  optimal  process  improvements  take 
place.  However,  the  dimensions  that  positively  influence  group  performance  have 
continued  to  be  a  matter  of  debate. 

This  debate  has  been  fueled,  at  least  in  part,  by  research  which  has  produced 
mixed  results.  This  thesis  looked  at  several  structural  and  social  contingencies  to  explore 
possible  explanations  for  the  mixed  results  found  in  GSS  research.  The  study  examined 
the  effect  of:  anonymity,  identification,  self-regulation,  and  facilitator  provided  feedback 
on  user  information  sharing  behavior,  quality  of  the  group  decision,  unequal  participation 
among  group  members,  and  user  attitudes  with  the  GSS  meeting.  The  results  of  the  study 
indicated  that  the  anonymity  theory  was  partially  supported  since  users  in  an  anonymous 
condition  provided  more  intellective  comments  than  any  other  condition.  In  addition,  the 
self-regulation  theory  was  partially  supported  since  users  were  found  to  have  more 
satisfaction  with  the  GSS  meeting  in  this  treatment. 
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INFLUENCE  OF  FEEDBACK  AND  COMMENT  LABELS 


ON  INFORMATION  SHARING  IN  A  COMPUTER  MEDIATED  COLLABORATIVE 

ENVIRONMENT 

I.  Introduction 

A  group  support  system  (GSS)  is  a  computer  mediated  collaborative  environment 
that  allows  groups  to  generate,  organize,  evaluate,  and  communicate  information  (Anson, 
Bostrom,  and  Bayard,  1995:189).  Proponents  of  this  technology  have  claimed  that  GSS 
groups,  when  compared  to  face-to-face  groups,  achieve  greater  idea  generation 
(Nagasundaram  and  Bostrom,  1995:89),  higher  decision  quality  (Zigurs  and  Kozar, 
1994),  and  increased  process  gains  such  as  satisfaction  and  learning  (Mejias  and 
Shepherd,  1997:141).  Empirical  results  of  GSS  effectiveness,  however  have  been  mixed. 
Some  studies  found  strongly  positive  results  when  using  a  GSS  over  traditional  meetings, 
while  others  found  mildly  positive,  mixed,  neutral,  or  negative  results  (Lam,  1997:195). 
Contextual  factors  that  differ  from  study  to  study  may  provide  a  potential  explanation  for 
these  varied  findings. 

Recent  studies  have  tried  to  address  these  mixed  effects  by  looking  at  the  social 
and  structural  dimensions  of  GSS  designs.  For  example,  research  by  Lea  (1998)  showed 
that  attributes  of  the  facilitator  and  the  system  design  could  influence  the  perceptions  and 
social  interactions  of  the  individual  group  members.  This  study  showed  group 
performance  and  attitudes  toward  the  group  and  the  technology  improved,  when  the 
members  of  the  group  thought  the  facilitator  was  fair  and  the  structure  prevented  the 
facilitator  from  being  biased  toward  one  member  of  the  group.  A  second  study  by 
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Herberlie  and  Tolbert  (1999)  evaluated  the  impact  of  facilitator  alignment,  co-location, 
and  video  intervention  on  the  efficacy  of  a  GSS  when  deployed  in  a  distributed 
environment.  Although  many  of  their  findings  confirmed  Lea’s  study,  Herberlie  and 
Tolbert  found  that  the  GSS  facilitator  might  not  need  to  be  physically  separated  from  the 
decision  group  to  ensure  perceptions  of  neutrality.  Each  of  these  previous  efforts  called 
for  further  investigation  of  other  contextual  factors  that  may  influence  GSS  processes  and 
outcomes.  The  goal  of  this  line  of  research  is  to  discover  GSS  meeting  processes  and 
designs  that  can  consistently  increase  the  quantity  and  quality  of  comments  and  ideas 
generated  in  a  meeting,  and  further  the  likelihood  of  finding  increased  decision  quality 
(George,  Easton,  Nunamaker,  and  Northcraft,  1990:400).  Given  the  conflicting  research 
on  GSS  effectiveness,  and  the  promising  results  of  recent  research  on  contextual  factors 
of  GSS  design,  this  topic  is  worthy  of  further  investigation. 

The  study  reported  here  is  one  of  four  theses  that  continued  the  systematic 
investigation  of  contextual  factors  concerning  various  aspects  of  GSS  use.  All  four 
theses  used  the  same  experiment  to  collect  data,  but  each  study  examined  a  different  area 
in  GSS  research.  One  study  evaluated  the  effect  that  varying  levels  of  anonymity  had  on 
user  participation  in  GSS  and  face-to-face  groups.  A  second  study  investigated  the 
interaction  of  individual  characteristics  and  anonymity  on  participation  in  a  GSS 
environment.  A  third  study  analyzed  the  relative  contribution  of  individual  ideation, 
individual  expertise,  and  team  expertise  for  GSS  and  face-to-face  groups.  The  final 
study,  reported  in  this  paper,  examined  the  effects  of  coincidental  feedback  through 
comment  labeling,  loss  of  anonymity  through  participant  labeling,  and  feedback  provided 
by  the  facilitator  on  the  quantity  and  quality  of  ideas  generated  in  a  GSS  environment. 
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In  a  meta-analysis  of  GSS  research,  only  seven  of  the  over  200  studies  have  been 
conducted  on  the  impact  of  providing  feedback  during  a  GSS  session  (Fjermestad  and 
Hiltz,  1999).  Yet,  feedback  is  a  critical  element  in  goal  directed  and  self-regulatory 
behavior.  When  feedback  is  understood,  accepted,  and  acted  upon  by  the  participant  it 
can  enhance  performance,  motivation  (Young  and  Kline,  1996),  and  learning 
(Hollenbeck,  Hedlund,  Ilgen,  Lapine,  and  Colquitt,  1998). 

Assessment  feedback,  is  defined  as  information  provided  to  an  individual 
concerning  some  aspect  of  their  performance  on  a  task  (Young  and  Kline,  1996:44).  This 
study  investigates  the  effects  of  two  distinct  types  of  feedback  on  individual  participation 
and  group  effectiveness.  Assessment  feedback  can  be  provided  to  users  after  task 
completion  showing  how  often  they  contributed  and  how  well  their  team  scored  on  the 
task.  This  feedback  will  allow  the  group  members  to  participate  more  evenly  so  the 
quality  of  the  group  decision  will  increase  on  future  tasks  given  to  the  group.  Coincident 
feedback  is  the  specific  information  received  by  participants  while  the  task  is  being 
performed.  One  form  of  coincidental  feedback  provided  in  group  support  systems  occurs 
through  comment  labeling.  Labeled  comments  provide  a  running  tally  on  the 
participation  of  group  members.  Individuals  can  use  coincidental  feedback  to  regulate 
their  own  efforts,  as  well  as  apply  normative  pressures  to  other  group  members. 

This  study  also  investigates  the  effects  of  loss  of  anonymity  on  individual 
participation  and  group  effectiveness.  Combining  participant  labels  with  comment 
labeling  effectively  removes  any  anonymity  usually  offered  to  GSS  participants.  Group 
members  can  use  the  labels  to  directly  attribute  participation  to  specific  individuals  in  a 
manner  very  similar  to  a  face  to  face  meeting. 
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1.1  Background 

The  goal  of  a  GSS  is  to  improve  the  group  decision-making  process  by  removing 
communication  barriers,  providing  techniques  for  structuring  decision  analysis,  and 
systematically  directing  the  pattern,  timing,  or  content  of  the  discussion  (Lam,  1997:194). 
In  general,  a  GSS  is  designed  to  minimize  or  overcome  process  losses  of  traditional  face- 
to-face  meetings  while  seeking  to  enhance  process  gains  (Heminger  and  Tung,  1993:33). 
The  structural  and  social  dimensions  of  specific  GSS  designs  might  influence  whether 
process  improvements  ever  take  place.  This  study  will  look  at  two  dimensions  in 
particular  to  determine  if  process  improvements  occur.  The  study  looks  at  the  structural 
dimension  in  terms  of  comment  and  participant  labeling,  and  the  social  dimension  in 
terms  of  assessment  feedback  provided  by  the  facilitator. 

1.1.1  Effects  Through  Comment  and  Participant  Labeling 

Keeping  users  anonymous  in  a  GSS  session  has  been  touted  as  one  of  the  keys  to 
overcoming  process  losses.  Process  losses  are  events  that  occur  during  meetings  that 
prevent  the  group  from  performing  to  their  optimal  level.  By  keeping  the  identity  of  GSS 
participants  unknown,  anonymity  reduces  the  fear  of  social  disapproval  and  evaluation, 
increases  the  number  of  ideas  generated,  and  lowers  inhibition  and  censorship 
(Pinsonneault  and  Heppel,  1998:89).  Research  has  also  shown  that  keeping  the  identity 
of  members  involved  in  the  GSS  unknown  can  bring  negative  results  to  the  meeting  (El- 
Shinnawy  and  Vinze,  1998:171).  One  of  the  effects  of  anonymity  is  decreased 
participation  through  social  loafing.  Research  on  group  process  loss  suggests  that 
accountability  is  one  of  the  key  factors  in  ensuring  that  team  members  contribute  to  the 
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best  of  their  ability  (Hollenbeck  and  others,  1998:273).  Users  may  participate  less 
because  they  feel  that  anonymity  will  protect  them  from  pressures  to  perform  by  other 
group  members. 

Anonymity  in  a  GSS  session  can  be  reduced  by  labeling  comments  and  labeling 
participants.  Through  the  use  of  comment  labels,  participants  receive  coincidental 
feedback  on  the  performance  and  contributions  of  other  group  members.  The  label  is  a 
structural  component  of  the  GSS  environment  since  the  information  is  contained  on  the 
system.  Reducing  anonymity  through  comment  labeling  is  likely  to  have  a  positive  effect 
on  group  processes.  Comment  labeling  provides  a  coincidental  source  of  feedback  to 
group  members  that  may  inhibit  social  loafing  effects  and  provide  valuable  information 
about  individual  contributions  to  all  group  members.  On  the  other  hand,  labeling  group 
participants  can  potentially  have  a  negative  effect  on  the  group  process.  Labeling 
participants  causes  the  loss  of  anonymity  that  inhibits  some  users  from  participating  in 
the  meeting.  This  study  will  examine  the  labeling  effect  to  determine  if  labeled  users  will 
contribute  more  ideas  to  reach  an  optimal  solution  to  a  group  problem. 

1.1.2  Facilitator  Effects  Through  Goal  Setting  and  Assessment  Feedback 

A  facilitator  is  commonly  used  in  a  GSS  meeting  to  direct  activities  before, 
during,  and  after  a  meeting  to  help  the  group  achieve  its  outcomes  (Niederman  and  Beise, 
1996:23).  The  facilitators’  role  has  been  identified  as  influential  to  the  GSS  meetings’ 
success  (Niederman  and  Beise,  1996:23).  To  enhance  user  participation,  a  group 
facilitator  typically  performs  specific  behaviors  during  a  meeting  such  as  managing 
conflict,  encouraging  broad  participation,  and  extending  idea  generation  (Anson  and 


5 


others,  1995:190).  This  study  will  investigate  an  additional  approach  to  enhance  user 
participation  through  providing  goals  and  assessment  feedback  to  participants. 

Feedback  and  goal  setting  have  been  found  to  be  important  management  tools 
because  they  serve  both  informational  and  motivational  functions  that  can  enhance  work 
behavior  (Early,  Northcraft,  Lee,  Lituchy,  1990:87).  The  tools  of  providing  goal  setting 
and  feedback  are  usually  combined  since  they  interact  to  achieve  greater  results  than 
when  they  are  used  separately.  Goal  setting  is  more  effective  and  usually  only  effective 
when  combined  with  feedback  since  it  allows  performance  to  be  tracked  in  relation  to  the 
goal  (Locke  and  Latham,  1990:241).  Feedback  allows  individuals  to  compare  their 
behavior  and  actions  against  predefined  targets  and  determine  whether  or  not  to  adjust 
their  actions  to  achieve  those  targets.  This  study  will  explore  whether  facilitator  provided 
goal  setting  and  assessment  feedback  given  in  a  GSS  meeting  will  produce  optimal 
meeting  results. 

1.2  Research  Applicability  to  the  United  States  Air  Force 

During  the  late  1980’s,  the  United  States  Air  Force  entered  a  period  where 
budgets  were  cut  and  personnel  were  reduced.  At  the  same  time,  Air  Force  senior 
leadership  emphasized  the  concept  that  Air  Force  personnel  must  work  smarter  and  do 
more  with  less.  During  this  period.  Air  Force  Logistic  Centers  faced  the  threat  of 
outsourcing  many  aircraft  maintenance  functions.  Also  with  the  potential  of  closing  two 
of  the  five  Air  Force  Logistic  Centers,  a  period  of  increased  competition  for  business 
within  the  Air  Force  Logistic  Centers  and  from  private  industry  began.  This  concept  of 
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competition  for  workloads  and  emphasis  on  cost  cutting  was  emphasized  to  logistic  units 
to  help  streamline  their  operations. 

Air  Force  logistics  units  were  introduced  to  the  concept  of  “Lean  Logistics” 
which  improves  an  operational  unit’s  capabilities  by  applying  modem  business  practices 
across  all  logistics  functions  and  processes.  At  the  Wamer-Robins  Air  Logistics  Center, 
the  Commander,  General  Hallin,  wanted  workers  to  find  ways  to  improve  processes,  save 
time  and  money  to  become  the  “Wal-Mart”  of  all  Logistic  centers  (Hallin,  1994). 
Logistics  centers  realized  that  they  needed  to  increase  communication  and  collaboration 
efforts  among  affected  parties,  which  include  senior  management,  logistics  and 
maintenance  personnel,  engineers,  and  government  contractors.  Group  support  systems 
were  identified  as  a  tool  to  enhance  meetings  and  the  decision-making  process. 

Group  support  systems  are  being  used  by  the  Acquisition  Support  Team  and  the 
Reengineering  division  at  the  Wamer-Robins  Air  Logistic  Center,  Robins  Air  Force  Base 
Georgia.  The  Acquisition  Support  Team  uses  a  GSS  to  assess  acquisition  risks  by 
performing  risk  reduction  analysis,  best  course  of  action,  and  collaborative  thinking  and 
development.  The  GSS  has  given  the  government  and  contractors  the  ability  to 
determine  requirements  and  develop  project  specification  documents.  A  GSS  allows 
parallel  communication  between  parties,  which  decreases  the  time  to  develop  a  project’s 
requirements  documents. 

For  example,  the  Acquisition  Support  Team  used  a  GSS  to  solve  a  dispute  over 
available  technology  for  the  next  generation  of  flight  line  heaters.  In  the  initial  project 
planning,  the  government  submitted  a  request  that  was  technologically  impossible  for  the 
contractor  to  deliver.  A  GSS  allowed  both  groups  to  work  together  to  determine  the  best 
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course  of  action.  The  GSS  permitted  users  to  speak  freely,  without  fear  of  retribution, 
allowing  a  consensus  to  be  reached  and  a  decision  made  in  a  timely  manner. 

The  GSS  facilitators  at  Robins  Air  Force  Base  state  it  takes  users  a  whole  day 
before  they  become  comfortable  with  the  system  and  understand  the  GSS  functions.  An 
entire  day  of  productivity  could  be  saved  if  users  could  be  trained  more  efficiently  so 
they  understood  the  purpose  of  the  GSS,  how  to  use  it,  and  what  was  expected  of  their 
role.  Not  only  would  valuable  time  be  saved,  but  also  reducing  a  meeting  by  one  day 
would  save  considerable  expense.  For  example,  the  GSS  facilitator  estimated  that  up  to 
90  people  participate  in  one  of  their  sessions  at  a  given  time.  The  cost  of  avoidance  in 
productivity  could  amount  to  tens  of  thousands  of  dollars  per  session.  An  additional 
factor  of  concern  is  the  motivation  of  team  members  to  use  a  technology,  regardless  of  its 
proposed  benefits,  where  the  costs  seem  so  high. 

The  GSS  facilitator  also  indicated  that  during  the  first  day  of  meetings,  he  spends 
most  of  his  time  answering  questions  and  concentrating  on  the  flow  of  the  meeting.  Like 
most  new  technology,  there  is  a  learning  curve  that  participants  must  overcome  in  order 
to  become  proficient  with  the  system.  The  development  of  a  familiarization  exercise 
could  help  people  climb  the  steep  learning  curve  in  a  relatively  short  time.  If  users  could 
quickly  understand  the  GSS,  the  facilitator  could  free  up  this  time  to  focus  on  other 
facilitator  duties.  This  extra  time  could  be  used  to  allow  the  facilitator  to  provide 
feedback  to  users  and  reach  a  more  efficient  and  higher  quality  decision.  This  study 
hopes  to  add  insight  into  the  configurations  and  procedures  for  a  GSS  that  could  make 
meetings  more  effective. 
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1.3  Summary 

A  GSS  is  a  combination  of  computer  hardware,  software,  and  human  facilitation 
that  provides  an  assortment  of  communication  and  decision  support  tools  intended  to 
increase  the  effectiveness  and  efficiency  of  decision-making  groups  (Lea,  1998).  The 
benefits  of  providing  coincidental  feedback  during  a  GSS  session  have  not  been 
examined  and  studying  the  optimal  combination  of  facilitator  provided  feedback  and  user 
labeling  has  yet  to  be  addressed  in  a  GSS  environment. 

Unclear  roles  of  GSS  facilitators  have  raised  awareness  that  facilitators  may  not 
be  doing  all  they  can  to  improve  the  meeting  process.  Facilitators  at  Wamer-Robins  Air 
Logistic  Center  are  looking  for  ways  to  assist  GSS  users  to  quickly  understand  the  GSS 
process  and  functions.  Providing  feedback  and  goals  to  users  may  accelerate  the  process 
and  allow  users  to  reach  a  better  group  decision. 

While  much  is  understood  about  the  impact  of  feedback  and  goal  setting  on 
traditional  face-to-face  groups,  little  is  known  about  the  impact  they  have  on  GSS  groups. 
This  study  will  shed  light  on  this  area  by  investigating  the  influence  of  feedback  and 
comment  labels  on  information  sharing  in  a  GSS  environment. 

This  research  will  investigate  the  interactive  effects  of  coincidental  feedback 
through  comment  labeling,  loss  of  anonymity  through  participant  labeling,  and  facilitator 
provided  assessment  feedback  on  group  participation,  decision  quality,  and  equal  group 
participation. 
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1.4  Sequence  of  Presentation 

Chapter  II  of  this  study  provides  a  review  of  the  relevant  literature  from  the  body 
of  GSS  research  with  emphasis  on  research  pertaining  to  the  independent  variables 
(coincidental  feedback  and  feedback)  studied  in  this  thesis.  Chapter  III  details  the 
methodology  used  to  conduct  the  research  for  this  study  and  the  results  of  this  study  are 
presented  in  Chapter  IV.  Finally,  Chapter  V  discusses  the  data  with  respect  to  the 
hypotheses  that  were  investigated  with  this  study.  Findings  are  presented  with  the 
conclusions,  limitations,  and  recommendations  for  future  GSS  research. 
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II.  Literature  Review 


2.1  Introduction 

In  the  1950’s,  researchers  began  to  intently  study  small  group  decision-making 
and  the  contribution  of  individual  team  member’s  actions  to  group  effectiveness.  This 
research  found  that  many  process  losses  occurred  in  groups.  Process  losses  such  as  social 
loafing,  evaluation  apprehension,  attention  blocking,  information  overload,  and 
dominance  did  not  allow  groups  to  perform  at  their  optimal  level.  Researchers  then 
began  to  look  for  new  ways  to  use  technology  to  overcome  these  problems  and  began 
experimenting  with  group  support  systems  to  help  minimize  or  eliminate  some  of  the 
process  losses.  After  much  experimentation,  group  support  systems  made  their  first 
appearance  in  literature  in  1982  where  the  benefits  of  a  GSS  over  traditional  face-to-face 
meetings  were  reported  (Fjermestad  and  Hiltz,  1999).  While  most  prior  GSS  research 
compared  the  outcomes  of  a  GSS  versus  traditional  face-to-face  meetings,  the  more 
recent  research  is  interested  in  specific  manipulations  of  a  GSS  to  make  meetings  more 
productive  and  efficient  (Zigurs  and  Kozar,  1994). 

2.2  GSS  Overview 

The  GSS  is  a  tool  that  organizations  can  use  to  reduce  the  previously  mentioned 
process  losses  that  are  often  associated  with  group  meetings.  The  main  objective  of  the 
GSS  is  to  improve  the  group  decision-making  process  by  removing  the  communication 
barriers,  structuring  the  group  interaction,  and  providing  analytical  tools  for  the  task  by 
using  technology  to  assist  the  meeting  (Hiltz,  Johnson,  and  Turoff,  1991:82). 
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The  technology  used  in  a  GSS  includes  a  networked  personal  computer  for  each 
group  member;  software  that  collects,  manipulates,  and  aggregates  the  individual  input 
contributed  by  each  member;  individual  monitors  and  a  projected  image  to  display 
individual  and  aggregated  information  (Zigurs  and  Kozar,  1994).  The  projected  image 
acts  as  a  focus  point  for  discussions.  While  technology  plays  an  important  role  in  a  GSS 
session,  a  human  facilitator  is  often  present  to  ensure  a  GSS  meeting  will  run  more 
efficiently.  A  facilitator  makes  sure  that  all  group  members  get  involved  in  the  early 
stages  of  the  meeting  and  will  also  manage  dominant  members  of  the  group  to  ensure 
equal  participation  from  all  members  (Clawson  and  Bostrom,  1993).  Most  research 
conducted  on  the  benefits  of  using  a  facilitator  in  a  GSS  meeting  show  that  facilitated 
GSS  groups  have  a  markedly  greater  likelihood  of  producing  favorable  effects  in 
comparison  to  nonfacilitated  groups  (Fjermestad  and  Hiltz,  1999;  Anson  and  others, 
1995:189).  The  potential  for  increased  productivity  have  made  group  support  systems  an 
increasingly  popular  tool  in  many  organizations. 

2.3  GSS  Benefits 

One  of  the  most  common  methods  to  solve  or  work  on  a  problem  is  for 
individuals  to  work  together  in  groups  to  arrive  at  an  accepted  solution.  There  is 
widespread  consensus  from  industry  experts  that  in  the  coming  years,  work  will  be 
performed  in  task-focused  teams,  rather  than  in  traditional  departments  or  individual 
members  of  the  organization  (Ancona  and  Nadler,  1989;  Drucker,  1988).  Group  support 
systems  are  already  being  used  today  for  many  different  business  applications  such  as 
business  process  reengineering,  requirement  elicitation,  process  and  data  model 
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development,  and  creative  idea  generation  (Nagasundram  and  Bostrom,  1995).  Many 
organizations  are  beginning  to  experiment  with  GSSs  to  realize  some  of  the  benefits  they 
provide,  while  other  organizations  are  increasing  their  usage  of  the  tool.  Group  support 
systems  have  been  a  success  in  organizations  with  some  realizing  an  average  labor 
savings  of  50-70%  and  a  reduction  in  project  cycle  time  averaging  90%  (Reinig,  Briggs, 
Shepherd,  Yen,  and  Nunamaker,  1996:171).  The  reports  of  the  dramatic  benefits  of  a 
GSS  should  be  viewed  as  genuine  and  not  exaggerated.  It  is  unlikely  that  anyone  would 
use  a  GSS  if  the  system  did  not  increase  group  performance  since  designing,  building, 
operating,  and  maintaining  a  GSS  is  quite  expensive  (George  and  others,  1990:398). 

A  GSS  meeting  may  be  preferred  over  a  traditional  face-to-face  meeting  since  it 
has  been  shown  that  GSS  meetings  tend  to  produce  more  comments  from  meeting 
participants  (Zigurs  and  Kozar,  1994).  The  quantity  of  ideas  that  are  generated  correlates 
very  highly  with  the  average  quality  of  ideas  produced  during  a  meeting  (Nagasundaram 
and  Bostrom,  1995:96;  Shepherd,  Briggs,  Reinig,  Yen,  and  Nunamaker,  1996:155). 
Allowing  group  members  to  exchange  multiple  ideas  is  important  since  information 
sharing  is  positively  related  to  the  quality  of  a  group  decision  (Mennecke  and  Valachich, 
1998:174).  Group  members  often  feed  off  other  individual’s  ideas,  or  piggyback,  which 
then  leads  to  more  comments  and  better  solutions.  When  groups  can  access  a  larger  pool 
of  information  about  a  problem,  it  will  potentially  enable  them  to  make  a  better  decision 
for  the  organization  (Dennis,  1996).  Therefore,  groups  have  the  potential  to  outperform 
individuals  on  a  task  due  to  the  synergy  that  is  generated  from  having  individuals  work 
together  (George  and  others,  1990:395). 
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While  generating  numerous  ideas  is  valuable,  groups  must  be  careful  to  give 
everyone  a  chance  to  participate  in  a  meeting  so  that  a  dominant  member  does  not  steer 
the  meeting  toward  just  one  point  of  view.  If  one  person  dominates  the  meeting,  that 
person  also  begins  to  obtain  power  and  subsequently  controls  the  tumtaking  of  who  is 
permitted  to  speak  in  the  meeting  (Hiltz  and  others,  1991:85).  In  a  traditional  face  to  face 
meeting,  only  one  person  can  speak  at  a  time  and  some  members  may  never  get  an 
opportunity  to  voice  their  ideas.  A  GSS  helps  minimize  the  effect  of  a  dominant  group 
member  by  reducing  production  blocking,  which  will  provide  all  users  an  equal 
opportunity  to  participate  in  the  meeting. 

2.2.1  Anonymity  and  Identification  in  a  GSS  Environment 

One  of  the  key  aspects  touted  about  a  GSS  to  help  reduce  process  losses  is  the 
anonymity  it  provides  to  users.  A  GSS  provides  different  levels  of  anonymity  depending 
on  the  options  chosen  by  those  involved  in  the  meeting. 

A  GSS  can  be  configured  so  that  none  of  the  comments  entered  during  the  session 
are  labeled  with  an  identifier.  This  would  provide  the  most  anonymity  to  the  participants 
during  the  session.  The  GSS  configuration  could  also  be  manipulated  to  allow  a  label 
with  a  generic  identifier  such  as  red,  green,  or  blue  to  indicate  the  submitter  of  each 
comment.  This  would  allow  group  members  to  associate  comments  to  a  participant  (e.g. 
red,  green,  or  blue),  but  still  provide  individuals  with  anonymity,  since  the  person  would 
not  be  linked  to  the  color  identifier.  The  GSS  could  take  away  anonymity  by  purposely 
labeling  each  member  of  the  meeting  with  their  comment  identifier. 
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Anonymity  has  been  found  to  be  one  of  the  most  interesting  and  important  aspects 
of  a  GSS,  yet  the  least  understood  (Valachich,  Dennis,  Jessup,  and  Nunamaker, 
1992:101).  For  example,  prominent  GSS  researchers  have  stated  that  little  is  really 
known  about  the  role  anonymity  plays  in  a  GSS  (George  and  others,  1990:411). 
Empirical  research  has  argued  for  the  need  to  investigate  the  effect  anonymity  provided 
in  a  GSS  has  on  group  processes  and  outcomes  (Jessup  and  George,  1997).  While  the  use 
of  a  GSS  has  been  shown  to  be  beneficial  there  are  mixed  results  from  GSS  studies 
showing  the  optimum  configuration  for  a  GSS.  In  particular  many  studies  have  shown 
conflicting  results  due  to  anonymity.  Anonymity  has  also  been  found  to  increase  critical 
comments,  and  to  have  a  negative  or  insignificant  effect  on  generating  comments,  group 
performance,  and  equality  of  participation  (Pinnonneault  and  Heppel,  1998:91).  In  a 
GSS  meta-analysis  looking  at  GSS  performance  by  comparing  anonymous  vs  identified 
groups,  twenty  out  of  thirty-three  experiments  showed  the  anonymous  condition  being 
significantly  better  than  the  labeled  condition.  However  labeled  conditions  were 
significantly  better  than  anonymous  conditions  ten  times  (Fjermestad  and  Hiltz,  1999). 
To  get  a  clearer  picture  of  what  is  really  occurring  during  a  GSS  session,  we  need  to  take 
a  closer  look  at  the  differing  schools  of  thought  on  labeling  persons  in  a  GSS  session. 

This  chapter  will  explore  possible  explanations  for  the  mixed  results  in  GSS 
research  by  looking  at  the  GSS  stmctural  and  social  contingencies.  The  structural 
contingencies  depend  on  the  way  the  GSS  configuration  is  set  up  for  the  users.  The 
chapter  will  look  at  three  competing  areas  of  thought  that  are  believed  to  contribute  to  the 
success  of  a  GSS:  anonymity,  identification,  and  self-regulation  provided  through 
coincidental  feedback.  A  discussion  on  the  social  contingencies  will  then  be  presented 
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that  demonstrates  how  a  facilitator  could  provide  goals  and  feedback  to  the  group  during 
the  GSS  session.  This  chapter  will  also  present  the  hypotheses  investigated  for  the 
research  contained  in  this  study.  By  testing  the  hypotheses  of  each  competing  theory,  this 
paper  hopes  to  provide  more  information  to  the  body  of  knowledge  concerning  the 
optimal  configuration  and  administration  of  a  GSS  session. 

2.3  Structural  Contingencies 
2.3.1  GSS  Anonymity 

A  GSS  has  been  linked  to  greater  task  orientation,  increased  decision  quality,  and 
significantly  increased  idea  generation  when  compared  with  a  traditional  face-to-face 
meeting  (Zigurs  and  Kozar,  1994;  Nagasundram  and  Bostrom,  1995:89).  There  are 
generally  three  accepted  components  that  contribute  to  improved  performance  in  a  GSS 
meeting  that  cannot  be  provided  in  a  traditional  face-to-face  meeting,  these  components 
are  parallelism,  group  memory,  and  anonymity.  (Nunamaker,  Dennis,  Valacich,  Vogel, 
and  George,  1991).  This  section  of  the  paper  will  focus  on  the  anonymity  component 
found  in  a  GSS. 

Anonymity  is  defined  as  the  identifibility  of  group  member  contributions  (Jessup 
and  Connolly,  1990).  This  includes  the  ability  of  group  members  to  correctly  identify  the 
source  of  messages  they  receive  and  the  destination  of  any  messages  they  send  to  other 
group  members  (George  and  others,  1990:398).  Working  in  a  group  might  inhibit  a 
person  who  anticipates  embarrassment,  a  hostile  environment,  conformity  pressures,  or 
other  punishments  from  contributing  a  unique  idea  (Collaros  and  Anderson,  1969).  Often 
individuals  may  not  participate  in  a  face-to-face  meeting  because  of  personality 
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characteristics  or  out  of  fear  of  voicing  their  opinion  in  front  of  superiors.  People  do  not 
want  to  appear  weak,  unintelligent,  or  upset  the  cohesiveness  of  the  group  and  the 
anonymity  provides  an  avenue  for  people  to  voice  their  differing  opinions  (Jessup  and 
Connolly,  1990).  Configuring  a  GSS  so  that  any  particular  comment  cannot  be  directly 
tied  back  to  the  submitter  should  help  reduce  any  rejection  fears.  With  anonymity 
assured  all  members  of  the  group  should  feel  more  comfortable  to  voice  their  opinion 
without  any  fear  of  retribution  or  embarrassment  from  other  members  of  the  group. 

An  anonymous  GSS  setting  removes  the  normative  cues  associated  with  position, 
status,  or  past  individual  behavior  (El-Shinnawy  and  Vinze,  1998).  Information  and 
comments  exchanged  therefore  may  not  be  rejected  solely  on  the  basis  of  individual 
member  characteristics  such  as  social  status  or  expertise  (Dennis,  Hilmer,  and  Taylor, 
1998:65).  Users  often  begin  to  feel  free  to  express  their  ideas  since  the  idea  will  be 
judged  on  its  own  merit,  not  on  who  has  suggested  the  idea.  Anonymity  encourages 
merit-based  evaluation  of  ideas  that  can  lead  to  more  honest  and  creative  input  since  it 
decreases  conformity  pressure  (Boiney,  1998).  Therefore  cognitive  resources  are  spent 
evaluating  the  idea  and  not  on  any  bias  a  person  may  have  against  the  sender  of 
information. 

Anonymity  has  been  studied  in  many  experiments  and  its  benefits  have  produced 
tangible  results  for  the  group.  In  a  short  summary,  anonymity  reduces  the  fear  of  social 
disapproval  and  of  evaluation,  lowers  user  inhibitions,  and  censorship.  Overall 
anonymity  has  been  shown  to  improve  participation  and  communication,  promote  more 
objective  and  honest  evaluation  of  contributed  ideas,  and  enhance  the  productivity  of 
groups  and  their  decision-making  process  (Connolly,  Jessup,  and  Valacich,  1990). 
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Hypothesis  1  Effects  Through  Anonymity 

Research  has  suggested  that  increasing  a  users  anonymity  will  increase  their 
participation  levels  since  anonymity  reduces  a  persons’  fear  of  rejection.  Anonymity 
should  also  increase  the  equality  of  participation  and  quality  of  decisions  since  it  allows 
the  maximum  amount  of  cognitive  resources  to  focus  on  the  information.  Participants 
should  be  willing  to  contribute  to  the  group  discussion  and  not  hold  back  any  ideas.  The 
participants’  main  focus  should  be  on  solving  the  problem  at  hand,  not  on  how  their 
comments  may  be  accepted  or  rejected  by  the  group.  Since  users  can  voice  their  opinions 
freely,  they  should  also  be  more  satisfied  with  the  entire  meeting  experience. 

Hypothesis  1A:  Anonymity  will  have  an  effect  increasing  user  information  sharing 
behavior. 

Hypothesis  IB:  Anonymity  will  increase  the  quality  of  decision  for  the  group. 

Hypothesis  1C:  Anonymity  will  discourage  unequal  participation  from  group  members. 
Hypothesis  ID:  Anonymity  will  encourage  positive  user  attitudes  with  the  GSS  meeting. 

2.3.2  GSS  Identification 

While  there  have  been  many  studies  discussing  the  benefits  of  anonymity  there 
has  also  been  studies  that  show  the  potential  pitfalls  anonymity  can  create.  Psychological 
research  on  interpersonal  processes  shows  that  anonymous  interaction  does  not  always 
result  in  positive  outcomes  (Jessup  and  George,  1997).  Anonymity  should  promote  more 
objective  and  honest  evaluation  of  ideas  and  improve  the  quality  of  decision-making  and 
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group  performance,  yet  it  also  induces  the  negative  effects  of  flaming  and  social  loafing 
(Pinsonneault  and  Heppel,  1998:91). 

2.3.2. 1  Flaming 

Flaming  is  where  group  members  begin  to  demonstrate  uninhibited  interaction 
and  begin  to  send  critical  comments  to  other  members  of  the  group.  Flaming  occurs 
when  anonymous  group  members  begin  to  lose  their  individuality  which  results  in  a 
reduction  of  normal  inhibitions,  enabling  group  members  to  engage  in  behavior  they 
would  not  normally  display  in  a  labeled  situation  (Jessup  and  Connolly,  1990).  Some 
GSS  groups  have  been  known  to  degrade  into  just  plain  silliness  and  do  not  focus  on  the 
task  at  hand  (Jessup  and  George,  1997).  Flaming  can  be  reduced  when  people  are 
identifiable  in  a  meeting  since  normative  behavior  is  more  likely  to  occur  when  users  are 
labeled  (Marx,  1999:105). 

2. 3. 2.2  Social  Loafing 

The  other  negative  effect  of  anonymity  is  social  loafing.  Social  loafing  is  the 
tendency  for  people  to  exert  less  effort  when  they  pool  their  efforts  toward  a  common 
goal  than  when  they  are  held  individually  accountable.  A  common  explanation  for  social 
loafing  is  that  there  is  an  inability  to  identify  individual  efforts  in  a  group  environment 
(Gagne  and  Zuckerman,  1999:525).  Social  loafing  increases  with  member  anonymity 
which  increases  with  group  size.  The  chance  of  detecting  and  identifying  social  loafing 
by  any  particular  member  decreases  as  group  size  increases  (Kerr  and  Braun,  1981:228). 
When  people  are  in  a  small  group  they  realize  they  are  more  identifiable  and  do  not  want 
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to  risk  letting  others  know  that  they  are  not  pulling  their  weight  in  a  meeting.  Literature 
on  social  loafing  indicates  that  accountability  is  one  of  the  essential  factors  in  ensuring 
each  member  of  the  group  contributes  to  the  best  of  their  ability  (Miles  and  Greenberg, 
1993;  Williams,  Harkins,  and  Latane,  1989). 

Social  loafing  can  occur  in  a  variety  of  tasks  including  those  that  require  physical 
effort  such  as  running  (Huddleston,  Doody,  and  Ruder,  1985)  or  cognitive  effort  such  as 
reacting  to  proposals  (Brickner,  Harkins,  and  Ostrom,  1986).  Studies  have  shown 
situations  where  members  who  are  identified  exert  greater  physical  or  mental  effort  than 
those  working  anonymously  (Jessup  and  Connolly,  1990).  One  study  found  that 
participants  in  a  tug-o-war  competition  pulled  less  hard  on  the  rope  when  they  believed 
that  other  people  were  pulling  with  them  (Ingham,  Levinger,  and  Peckman,  1974). 
Another  study  involving  a  GSS  found  that  participants  worked  harder  when  co¬ 
participants  could  evaluate  the  individual  performance  of  group  members  in  an  idea¬ 
generating  task  (Shepherd  and  others,  1996:158).  These  are  just  a  few  examples  that 
show  how  increased  identification  can  be  an  important  mediator  in  social  loafing. 

We  often  believe  that  working  together  should  inspire  group  members  to 
maximize  the  group’s  potential  and  work  harder  to  achieve  positive  outcomes.  However, 
many  times  the  benefits  that  group  interaction  provides,  is  not  enough  to  arrive  at  the 
optimal  solution.  Often  the  information  possessed  by  individual  members  is  not  shared 
with  other  members  of  the  group.  Another  downfall  of  working  collectively  is  that  it 
lessens  the  possibility  of  being  evaluated,  which  encourages  people  to  participate  in 
social  loafing  (Charbonnier,  Huguet,  Brauer,  and  Montiel,  1998:331).  When  participants 
were  led  to  believe  that  their  outputs  could  be  individually  identified,  social  loafing  was 
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eliminated  (Williams  and  others,  1981).  Making  an  individual’s  performance  more 
identifiable  reduces  the  sense  of  the  person  being  lost  in  the  crowd  and  causes  people  to 
work  harder  (Swain,  1996:338).  Researchers  found  that  anonymous  subjects  loafed  in 
some  GSS  experiments  and  exerted  less  effort  than  participants  who  were  identified  with 
a  label.  Identified  subjects  also  typed  more  comments  than  those  in  an  anonymous 
situation  (Jessup  and  George,  1997). 

When  people  are  labeled,  they  are  identified  to  other  members  of  the  group  which 
now  makes  them  accountable  to  the  group  for  their  actions  (Marx,  1999:105).  By 
labeling  users  in  a  GSS  meeting,  the  label  creates  higher  levels  of  accountability  for  those 
involved  in  the  meeting  (Hollenbeck  and  others,  1998:280).  As  more  verbal  and  non¬ 
verbal  cues  are  removed  from  the  group  process  there  begins  a  loss  of  social  presence 
(Short,  Williams  and  Christie,  1976;  Rice  1993).  That  is,  people  become  more  like 
objects  and  not  like  real  human  beings.  The  result  of  the  deindividualiztion  is  often  the 
anti-social  behavior  previously  mentioned  that  could  be  detrimental  to  the  effectiveness 
of  the  group. 

Hypothesis  2  Effects  Through  Identification 

Since  anonymity  and  identification  are  opposite  factors.  Hypothesis  1  and  2 
predict  the  opposite  results.  The  discussion  on  identification  suggests  that  increasing 
identification,  thereby  reducing  the  anonymity,  increases  participation  by  producing 
social  pressures  on  each  member  to  do  their  fair  share.  Anonymity  encourages  social 
loafing  and  decreased  participation.  Identification  introduces  social  pressures  to 
participate  in  the  group.  Increased  participation  increases  the  chance  that  good  ideas  will 
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be  generated  and  higher  quality  decisions  will  be  achieved.  Social  pressures  also  will 
encourage  equal  participation  since  people  will  want  to  contribute  to  the  group  and  not  be 
singled  out  as  a  non-contributor.  Since  an  optimal  solution  should  be  found  in  an 
identified  group,  the  group  will  be  satisfied  with  the  overall  GSS  experience. 

Hypothesis  2A:  Identification  will  have  an  effect  increasing  user  information  sharing 
behavior. 

Hypothesis  2B:  Identification  will  increase  the  quality  of  decision  for  the  group. 
Hypothesis  2C:  Identification  will  discourage  unequal  participation  from  group 
members. 

Hypothesis  2D:  Identification  will  encourage  positive  user  attitudes  with  the  GSS 
meeting. 

2.3.3  Coincidental  Feedback 

When  users  receive  coincidental  feedback  they  begin  to  become  self-aware  of 
their  performance.  The  user  begins  to  compare  one’s  perceived  performance  against  a  set 
standard.  When  people  realize  that  a  discrepancy  exists  between  their  performance  and  a 
stated  goal  they  are  faced  with  three  options  according  to  the  Carver  and  Scheier  theory: 
adhere  to  the  standard  and  exert  greater  effort,  lower  the  standard  and  continue  to  exert 
effort  or  withdraw  from  the  task  (Kanfer,  1990:134). 

Self-regulation  basically  involves  altering  ones  own  response  against  a  stated  goal 
(Muraven,  Baumeister,  and  Tice,  1999).  Bandura  and  Kanfer  view  self-regulation  as 
consisting  of  three  major  components:  self-observation,  self-evaluation,  and  self-reaction 
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(Kanfer,  1990:131).  Self-observation  is  the  attention  given  to  specific  aspects  of  one’s 
own  behavior  that  can  occur  from  external  or  internal  prompts  (Kanfer,  1990:132). 
These  prompts  provide  information  to  the  user  about  the  consequences  of  their  actions. 
So  through  self-monitoring  a  person  can  gain  knowledge  about  how  they  are  performing 
on  a  task.  The  next  component  is  self-evaluation.  Feedback  affects  self-evaluation  in 
that  it  provides  a  reference  point  for  the  performance  comparison.  A  person  can  compare 
their  performance  against  the  stated  goal  and  the  user  can  evaluate  their  performance. 
This  is  important  since  many  times  a  user  will  feel  positive  about  their  performance  when 
the  feedback  mechanism  indicates  that  they  are  performing  sub-par.  This  allows  the  user 
to  make  corrections  to  their  performance.  The  final  component  is  self-reaction  and  this  is 
the  internal  response  that  occurs  in  response  to  self-evaluation  (Kanfer,  1990:132).  If  the 
self-evaluation  indicates  that  a  person’s  performance  is  consistent  with  the  stated  goal 
then  this  yields  satisfaction,  while  if  performance  is  not  meeting  the  goal  then  this  yields 
dissatisfaction. 

For  example,  swimmers  are  often  used  to  demonstrate  self-regulation  since  they 
must  be  able  to  delay  immediate  gratification  and  improve  their  toleration  of  unpleasant 
events  in  order  to  achieve  their  long-term  goal  of  succeeding.  During  competition 
swimmers  use  self-regulation  to  pace  themselves  against  their  opposition.  They  are 
aware  of  the  other  swimmers’  performance,  yet  they  are  also  aware  of  their  own  abilities 
and  strategies  to  succeed  in  the  event.  The  coincidental  feedback  received  by  the 
swimmers  allows  them  to  take  all  of  these  inputs  and  determine  the  pace  that  is  needed  to 
achieve  a  successful  outcome  (Anshel  and  Porter,  1996). 
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Coincidental  feedback  will  also  provide  group  members  with  information  on  how 
often  they  are  participating  in  a  task.  Decision-making  is  often  more  difficult  with  the 
dominance  of  one  individual  since  it  causes  some  members  of  the  group  to  be  reluctant  to 
participate  (Steiner,  1972).  Equality  of  participation  in  a  meeting  has  been  shown  to 
increase  the  performance  of  the  group  as  well  as  increase  the  number  of  ideas  that  are 
generated  in  the  meeting  (Dennis,  1991;  Nunamaker,  Vogel,  and  Konsynski,  1989). 
Equal  rates  of  participation  may  generate  alternative  solutions  that  may  not  have  been 
generated  otherwise.  These  solutions  may  be  more  attractive  to  the  group  than  the 
solution  suggested  by  the  more  dominant  members  of  the  group  (George  and  others, 
1990:396). 

Comment  labeling  provides  feedback,  which  is  important  to  the  speed  and 
effectiveness  of  the  communication  process  (Dennis  and  Kinney,  1998:260).  If  GSS 
members  can  associate  comments  to  participants,  they  can  integrate  pieces  of  information 
received  at  different  times.  Labeling  thus  helps  minimize  the  time  of  understanding  so 
the  amount  of  time  to  complete  a  task  is  shortened  (Dennis  and  Kinney,  1998:260).  In 
addition,  the  group  can  have  specific  idea  exchange  so  they  can  defend  and  criticize 
specific  ideas  (Valacich  and  others,  1992:105). 

Facilitators,  which  have  already  been  identified  as  a  key  to  GSS  success,  believe 
that  providing  coincidental  feedback  will  be  a  benefit  to  the  GSS  and  will  lead  to  superior 
results  (Kelly  and  Bostrom,  1998:36).  Participants  can  use  the  coincidental  feedback  to 
develop  an  evaluation  of  a  participants’  competence  that  will  influence  their  choice  to 
align  with  his  or  her  ideas.  Also  people  may  notice  that  they  are  dominating  the  meeting 
and  may  choose  to  contribute  less  so  others  may  become  more  involved.  Participants 
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may  also  realize  that  one  member  or  members  of  the  group  are  not  contributing  ideas  to 
the  solution  of  a  problem,  and  then  apply  normative  pressure.  Labeling  provides  the 
feedback  that  could  allow  the  group  to  encourage  others  to  begin  participating  with  the 
group. 

2.3.3. 1  Credibility  Effects 

Labeling  users  in  a  GSS  provides  a  powerful  effect  on  individual  credibility. 
Comment  labeling  allows  participants  to  associate  comments  with  a  single  source.  The 
sum  of  single  source’s  comments  can  then  be  used  to  facilitate  an  estimate  of  his  or  her 
credibility.  If  a  GSS  user  types  in  a  comment  but  the  comment  cannot  be  traced  to  a 
specific  person,  how  do  other  group  members  know  that  the  comment  is  feasible  and 
technologically  valid?  Credibility  has  been  found  to  be  an  important  factor  to  the 
acceptance  and  processing  of  a  message  (Petty  and  Cacioppo,  1986).  In  theory,  people 
who  are  anonymous  could  make  up  information  and  not  be  held  responsible  for  their 
actions.  People  could  not  challenge  the  information  of  the  contributor  since  it  is  more 
difficult  to  immediately  challenge  a  comment  that  is  provided  in  an  anonymous  GSS 
setting  (Dennis,  1996).  This  is  particularly  important  when  the  information  being 
presented  or  discussed  in  the  GSS  is  ambiguous  or  difficult  to  process  (Petty  and 
Cacioppo,  1986). 
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Hypothesis  3:  Effects  through  Coincidental  Feedback 

Coincidental  feedback  suggests  that  comment  labeling  provides  additional 
information  that  helps  people  pace  themselves  and  integrate  ideas  by  both  source  and 
content.  The  comment  labels  are  important,  not  the  relative  level  of  anonymity.  The 
labels  allow  users  to  self-regulate  their  input,  which  allows  them  to  know  the  appropriate 
number  of  comments  to  submit.  Since  participants  have  more  information  available  to 
them,  the  group  should  generate  an  optimal  solution.  Coincidental  feedback  should  help 
create  more  balanced  participation  among  members  since  members  are  self-aware  of  their 
performance.  Coincidental  feedback  should  also  increase  satisfaction  with  the  GSS 
meeting  since  it  eases  integration  of  information.  So  the  most  information  is  comment 
and  person,  but  we  do  not  know  if  person  labels  have  a  positive  or  negative  effect,  which 
is  contradictory  from  Hypothesis  1  and  2. 

Hypothesis  3A:  Coincidental  feedback  will  have  an  effect  increasing  user  information 
sharing  behavior. 

Hypothesis  3B:  Coincidental  feedback  will  increase  the  quality  of  decision  for  the  group. 
Hypothesis  3C:  Coincidental  feedback  will  discourage  unequal  participation  from  group 
members. 

Hypothesis  3D:  Coincidental  feedback  will  encourage  positive  user  attitudes  with  the 
GSS  meeting. 
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2.4  Social  Contingencies 

2.4. 1  Facilitator  Effects  Through  Goal  Setting  and  Feedback 

One  of  the  most  common  motivational  techniques  to  increase  productivity  is  to 
provide  workers  with  goals.  Simply  defined,  a  goal  is  what  an  individual  is  trying  to 
accomplish,  it  is  the  object  or  aim  of  the  action  (Locke,  Shaw,  Saari,  and  Latham, 
1981:126).  Goals  enhance  performance  by  increasing  effort,  persistence,  directing 
attention,  and  improving  strategy  formulation  (Locke  and  others,  1981).  Goals  also 
become  a  motivator  for  people  since  it  challenges  them  to  improve  their  skill  level  and 
prove  their  competence  to  authority  figures  (Locke  and  Latham,  1990:241).  Over  400 
studies  have  been  conducted  on  goal  research  and  the  results  indicate  that  specific 
difficult  goals  lead  to  better  performance  than  specific  easy  goals,  vague  goals,  or  no 
goals  at  all  (Locke  and  Latham,  1990:240).  These  studies  also  indicate  that  goals  have 
little  effect  on  productivity  unless  they  are  associated  with  feedback  (Locke  and 
Latham,  1990:192). 

Feedback  is  often  given  to  show  progress  in  relation  to  the  specified  goal  (Locke  and 
others,  1981:133).  It  can  be  used  as  a  motivator,  an  information  giver,  an  explainer,  and 
an  idea  generator  (Zigurs  and  Kozar,  1994).  Feedback  provides  both  informational  value 
in  terms  of  promoting  learning  and  motivational  value  in  terms  of  promoting  effort 
(Ilgen,  Major,  Hollenbeck,  and  Raising,  1979;  Taylor,  Fisher,  and  Ilgen,  1984).  Most 
feedback  research  has  shown  that  performance  can  be  improved  by  providing  specific 
task-relevant  feedback  (Ammons,  1956;  Ilgen  and  others,  1979;  Kopelman,  1986)  and  it 
has  positive  effects  on  team  decision-making  accuracy  (Hollenbeck  and  others, 
1998:278).  The  performance  increases  are  possible  since  feedback  provides  users  the 
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type,  extent,  and  direction  of  errors  so  problems  can  be  corrected  (Becker,  1978:428). 
While  giving  a  goal  or  feedback  separately  has  shown  to  produce  positive  results,  most 
researchers  believe  that  there  is  an  interactive  effect  of  the  difficult  goal  and  the  feedback 
that  can  significantly  affect  performance  (Becker,  1978,  428). 

A  specific,  difficult  goal  combined  with  feedback  has  been  shown  to  increase 
productivity  among  workers.  (Locke  and  others,  1981:134;  Becker,  1978:428).  Subjects 
who  were  given  a  difficult  or  moderately  difficult  goal  performed  much  better  if  they 
received  feedback  than  those  groups  that  did  not  receive  feedback  (Frost  and  Mahoney, 
1976).  Feedback  and  goal  setting  work  hand  in  hand  to  achieve  superior  performance.  If 
people  are  only  given  feedback  then  they  have  no  goal  or  level  to  achieve.  If  people  have 
a  difficult  goal  but  no  information  is  given  on  how  well  they  performed,  then  there  is  no 
comparison  to  the  goal.  People  do  not  know  whether  they  have  met  the  goal,  surpassed 
the  goal,  or  has  failed  to  reach  the  standard,  so  the  feedback  becomes  irrelevant  (Becker, 
1978:429).  Without  feedback,  people  have  no  way  of  knowing  if  they  should  increase  or 
decrease  their  level  of  effort.  The  goal  and  feedback  provide  a  self-assessment  that 
allows  a  person  a  basis  for  adjusting  their  level  of  effort  (Earley  and  others,  1990:88).  A 
goal  plus  feedback  has  been  shown  to  produce  more  benefits  than  providing  a  goal  and 
no  feedback  (Schunk  and  Swartz,  1993).  It  is  the  joint  effect  of  goal  setting  and  feedback 
that  can  create  a  successful  outcome.  Two  research  studies  can  be  looked  at  to 
demonstrate  that  goal  setting  without  feedback  will  produce  a  negative  effect  on 
performance. 

For  example  in  a  study  that  looked  at  consumer  electrical  usage,  the  treatment  group 
that  received  a  goal  and  feedback  did  much  better  at  conserving  energy  than  the  group 
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that  received  a  goal  and  no  feedback.  This  study  helps  demonstrate  the  importance  of  the 
presence  of  providing  both  a  goal  and  feedback  toward  obtaining  optimal  performance 
(Becker,  1978:431). 

Another  study  conducted  at  Emery  Air  Freight  found  that  when  workers  received 
reports  of  their  results  (feedback)  and  a  goal,  they  performed  better  than  those  who 
received  no  information  of  their  work  results  (Locke  and  others,  1981: 134).  Interestingly 
those  who  received  no  goal  and  no  feedback  performed  just  as  well  as  those  who  received 
the  goal  without  feedback.  This  study  emphasizes  the  importance  of  receiving  both  a 
goal  and  feedback  if  management  wants  to  see  dramatic  increases  in  productivity. 

One  method  of  feedback  that  could  be  provided  in  a  GSS  is  to  compare  a  person’s 
individual  result  with  other  group  member’s  scores  and  against  a  known  average.  Studies 
have  shown  that  when  individuals  are  allowed  to  view  these  various  information  inputs 
that  it  heightens  the  competitive  nature  of  the  task  and  motivates  workers  to  perform  well 
(Williams  and  others,  1981:310).  People  are  often  curious  as  to  how  they  perform 
against  the  average  and  how  they  stack  up  against  other  people  in  a  group.  If 
participation  feedback  is  provided  to  users  during  the  GSS  session,  users  may  feel  more 
comfortable  with  their  own  participation  levels. 

If  people  can  receive  feedback  about  how  they  compare  with  other  group 
members  they  begin  to  participate  in  a  phoneme  called  social  comparison.  Social 
comparison  is  where  members  of  the  group  begin  to  match  their  performance  to  the  rate 
of  people  working  around  them.  For  example,  if  the  group  is  working  at  a  high  level  of 
effort  individual  members  tend  to  work  at  a  high  level  of  effort  as  well  (Goethals  and 
Darley,  1987).  Studies  have  shown  that  social  comparison  is  an  effective  tool  to  combat 
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social  loafing  in  groups  (Shepherd  and  others,  1996:169).  It  has  been  suggested  that 
feedback  can  play  an  important  role  in  group  decision-making  and  this  may  be  an 
important  direction  for  GSS  research  to  increase  productivity  and  fight  some  of  the 
negative  effects  found  in  a  GSS  (Sengupta  and  Te’eni,  1991:631). 

Hypothesis  4:  Effects  Through  Goal  Setting  and  Feedback 

Facilitator  provided  goals  and  feedback  suggests  that  group  members  will 
generate  more  comments  since  they  have  access  to  information  on  how  well  they 
performed  on  the  previous  task.  Since  participants  have  more  information  available  to 
them,  the  group  should  generate  an  optimal  solution.  Facilitator  provided  feedback 
should  create  more  balanced  participation  among  members  since  members  are  aware  of 
their  performance  vs  the  rest  of  the  group.  Facilitator  provided  feedback  should  also 
increase  satisfaction  with  the  GSS  meeting  since  users  now  possess  the  most  information 
available  to  them  in  a  GSS  setting. 

Hypothesis  4A:  Facilitator  provided  goal  setting  and  feedback  will  have  an  effect 
increasing  user  information  sharing  behavior. 

Hypothesis  4B:  Facilitator  provided  goal  setting  and  feedback  will  increase  the  quality  of 
decision  for  the  group. 

Hypothesis  4C:  Facilitator  provided  goal  setting  and  feedback  will  discourage  unequal 
participation  from  group  members. 

Hypothesis  4D:  Facilitator  provided  goal  setting  and  feedback  will  encourage  positive 
user  attitudes  with  the  GSS  meeting. 
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2.5  Interactive  Effects  of  Feedback  and  Anonymity/Identification 


In  the  preceding  paragraphs,  the  paper  discussed  the  potential  outcomes  that  may 
occur  when  participants  are  labeled/not  labeled  or  if  they  receive/do  not  receive  feedback. 
This  section  will  list  several  hypotheses  that  may  occur  when  these  items  interact. 

2.5.1  Interaction  of  Anonymity  and  Goal  Setting/Feedback 

Hypothesis  5:  Effects  Through  Anonymity  and  Goal  Setting/Feedback  Interaction 

If  Hypothesis  1  is  true,  and  anonymity  provides  positive  effects  by  reducing  fear 
of  rejection,  then  the  presence  of  assessment  feedback  could  produce  negative  results. 
When  users  believe  that  they  are  anonymous  in  a  GSS  meeting,  they  will  be  surprised  to 
discover  that  in  fact  their  comments  and  actions  are  not  anonymous.  They  will  realize 
that  they  are  being  observed  and  monitored  by  the  facilitator.  Users  will  discover  this 
when  they  are  presented  with  the  facilitator  feedback.  Since  participants  now  know  that 
their  identity  will  not  remain  anonymous,  they  will  participate  less  in  the  next  exercise 
since  they  have  lost  their  freedom  to  say  whatever  they  want.  Their  perception  of 
anonymity  is  different  from  the  actual  anonymity  being  provided.  Now  they  do  not  want 
to  risk  embarrassment  from  the  facilitator.  Since  many  users  will  feel  that  they  have  been 
lied  to  about  their  identity  status  many  will  not  have  a  pleasant  reaction  to  the  GSS 
meeting. 


31 


Hypothesis  5A:  Interaction  of  anonymity  and  facilitator  provided  goal  setting  and 
feedback  will  have  an  effect  discouraging  user  information  sharing  behavior. 

Hypothesis  5B:  Interaction  of  anonymity  and  facilitator  provided  goal  setting  and 
feedback  will  encourage  negative  user  attitudes  with  the  GSS  meeting. 

2.5.2  Interaction  of  Identification  and  Goal  Setting/Feedback 

Hypothesis  6:  Effects  Through  Identification  and  Goal  Setting/Feedback  Interaction 

Hypothesis  2  however  suggests  that  anonymity  could  have  a  negative  rather  than 
a  positive  effect  due  to  flaming  or  social  loafing.  If  this  hypothesis  proves  true,  then 
assessment  feedback  should  provide  a  synergistic  effect.  When  participants  who  have 
been  identified  in  the  GSS  session  receive  feedback  and  goal  setting  from  the  facilitator, 
they  should  begin  to  increase  their  participation  levels.  Participants  will  be  surprised  to 
discover  that  the  facilitator  has  actually  been  monitoring  the  events  that  transpire  over  the 
GSS.  The  feedback  will  let  the  participants  know  that  someone  is  watching  their 
performance  and  this  should  be  extra  motivation  to  participate.  As  a  result,  social  loafing 
should  be  even  less  evident  in  the  next  exercise  given  to  the  users,  and  flaming  if  present 
should  diminish. 

Hypothesis  6A:  Interaction  of  identification  and  facilitator  provided  goal  setting  and 
feedback  will  have  an  effect  increasing  user  information  sharing  behavior. 

Hypothesis  6B:  Interaction  of  identification  and  facilitator  provided  goal  setting  and 
feedback  will  discourage  unequal  participation  from  group  members. 
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2.5.3  Interaction  of  Coincidental  Feedback  and  Goal  Setting/Feedback 

Hypothesis  7:  Effects  Through  Coincidental  Feedback  and  Goal  Setting/Feedback 
Interaction 

When  participants  receive  coincidental  feedback  combined  with  goals  and 
feedback  provided  by  the  facilitator,  the  information  provided  to  group  members  should 
be  redundant  concerning  unequal  participation.  Group  members  should  already  be  aware 
of  the  rates  of  participation  and  the  extra  information  will  just  reinforce  their  beliefs.  The 
groups  that  had  not  received  coincidental  feedback  however  will  be  helped  by  the 
additional  information  and  should  begin  to  regulate  their  participation  levels. 

Hypothesis  7A:  Groups  who  do  not  receive  assessment  feedback  will  continue  to  show  a 
difference  based  on  the  presence  of  coincidental  feedback  through  comment  labels. 

2.9  Summary 

This  chapter  discussed  two  different  contingencies,  social  and  stmctural,  that  may 
explain  the  mixed  results  that  are  often  found  in  GSS  research.  Having  users  anonymous 
or  identified  during  a  GSS  meeting  are  competing  hypothesis  that  predict  opposite  effects 
on  the  outcome  variables.  The  research  design  that  will  be  used  in  this  study  is  important 
since  all  competing  hypotheses  can  be  evaluated  during  one  experiment  where  the 
anonymity  level  will  be  manipulated.  Once  all  data  is  collected  we  will  be  able  to 
compare  the  different  experimental  treatments  and  determine  through  statistical  analysis 
which  theory  or  theories  are  supported  by  the  data. 
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III.  Methodology 


3.1  Introduction 

As  stated  in  Chapter  One,  this  study  evaluated  the  impact  of  feedback  and 
coincidental  feedback  on  the  quantity  of  idea  generation  and  the  quality  of  decision¬ 
making  in  a  GSS  setting.  Chapter  Two  looked  at  the  previous  research  literature 
concerning  the  impact  of  feedback  and  coincidental  feedback  provided  by  labeling  on 
group  performance.  This  chapter  describes  how  data  were  collected,  computed,  and 
statistically  analyzed  to  test  the  hypothesized  relationship  between  the  independent 
variables  of  feedback  and  coincidental  feedback. 

3.2  Experimental  Design 

For  the  purpose  of  this  study,  two  widely  used  group  decision-making  tasks,  the 
Moon  Scenario  (Appendix  A)  and  the  Desert  Scenario  (Appendix  B),  were  used  to 
investigate  the  main  and  interactive  effects  of  anonymity,  coincidental  feedback,  and 
assessment  feedback  on  the  perceptions,  attitudes,  and  subsequent  behavior  (quantity  of 
comments  generated  and  quality  of  decision)  of  GSS  decision  making  groups.  These 
types  of  intellective  tasks  are  often  used  in  group  decision-making  research  (Bluedom, 
Turban,  and  Love,  1999:280).  In  each  manipulation,  four  subjects  interacted  as  a  team  to 
solve  the  given  scenario  using  a  GSS.  The  group  size  of  four  is  considered  appropriate 
since  the  average  number  of  people  attending  a  decision-making  meeting  is  usually  less 
than  five  (Lam,  1997:199). 
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Individuals  were  randomly  assigned  to  four-person  team  and  each  team  was 
randomly  assigned  to  one  of  the  treatments.  The  factorial  design  crossed  the  presence  of 
feedback  and  the  type  of  labeling  given  to  subjects.  Subjects  were  randomly  assigned  to 
one  of  six  cells  manipulated  feedback  (feedback  absent,  feedback  present)  and  labeling 
(GSS  meeting  with  no  labels,  GSS  meeting  with  only  comments  labeled,  and  GSS 
meeting  with  comments  and  users  labeled  via  a  placard). 

3.3  Subjects 

The  208  subjects  required  in  this  study  were  predominately  United  States  Air 
Force  Company  Grade  Officers  (Lieutenants  and  Captains)  who  were  either  graduate 
students  at  the  Air  Force  Institute  of  Technology,  students  at  the  Air  Force  Basic 
Communications  Officer  Training  (BCOT)  located  at  Keesler  Air  Force  Base 
Mississippi.  Roughly  25%  were  Air  Force  Reserve  Officer  Training  Corps  Cadets 
(college  Freshmen  through  college  Seniors).  The  age  of  subjects  averaged  25  years  old. 
Most  subjects  possessed  at  least  a  bachelor’s  degree  and  felt  comfortable  using  a 
computer.  The  study  used  both  male  and  female  subjects  since  gender  is  not  a  factor  in 
the  experiment.  See  table  below  for  more  complete  breakdown  of  demographics. 
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Table  1  Demographic  Percentages  of  Participants 


Education  Level 

Computer  Use 

Years  of  Use 

Hours  a  Week 

High  School 

1% 

Less  than  1 

0% 

0-10 

17% 

Some  College 

25% 

1-5 

19% 

11-20 

33% 

Bachelors 

46% 

6-10 

43% 

21-30 

27% 

Some  Graduate 

21% 

More  than  10 

38% 

More  than  30 

23% 

Graduate 

6% 

Gender 

Age 

Marital 

Status 

Male 

79% 

Low 

17 

Married 

44% 

Female 

21% 

High 

53 

Single 

56% 

Mean 

25.8 

3.4  Equipment  and  Facilities 

During  the  experiment,  the  subjects  were  placed  in  different  rooms,  one  called  a 
task  room  and  the  other  called  the  break  room.  In  the  break  room,  subjects  completed 
questionnaires  and  the  initial  scenarios  individually  with  paper  and  pencil.  The  break 
room  was  also  used  to  provide  subjects  overall  task  instructions,  goal  and  feedback 
information,  and  the  final  debrief.  The  task  room  was  the  location  where  the  individuals 
worked  as  a  group  to  solve  the  task  using  the  GSS. 

GSS  sessions  were  conducted  using  a  mobile  GSS  environment.  The  thesis  team 
configured  the  GSS  environment  at  each  experiment  location.  Care  was  taken  to  ensure 
the  room  layout  and  equipment  matched  as  closely  as  possible  for  each  location.  A  total 
of  six  Pentium  based  computers  and  one  server  were  configured  with  GroupSystems 
software  running  on  a  Windows  95  operating  system.  An  In-focus  machine  allowed 
subjects  to  view  the  results  of  the  experiment  on  an  overhead  screen. 
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3.5  Experiment  Manipulations 

Three  experiment  manipulations  were  used  during  the  study:  feedback, 
coincidental  feedback  provided  through  labeling,  and  lack  of  anonymity  provided 
through  person  labeling.  To  manipulate  feedback,  subjects  either  received  a  goal  and 
feedback  information  or  only  received  a  goal  after  completion  of  the  first  group  scenario. 
To  manipulate  labeling  and  anonymity,  subjects  were  randomly  assigned  to  a  particular 
treatment  where  comment  labeling  was  absent,  was  present,  or  comment  and  user 
labeling  was  present.  Experiment  manipulation  checks  were  included  in  the  post-test 
survey  given  to  all  participants  and  are  attached  in  Appendix  C.  In  order  to  determine 
manipulation  effectiveness,  the  means  from  groups  who  received  the  manipulation  of 
feedback  and  labeling  were  compared  to  those  groups  that  did  not  receive  the 
manipulation.  An  Analysis  of  Variance  (ANOVA)  test  was  conducted  to  compare  the 
difference  between  the  means  of  the  groups.  The  results  of  the  three  manipulation  checks 
were  successful  and  are  described  in  the  sections  that  follow. 

3.5.1  Feedback  Manipulation 

Feedback  was  successfully  manipulated  by  providing  subjects  verbal  and  visual 
feedback  after  the  first  scenario  was  completed.  Subjects  either  received  feedback  and  a 
goal  or  just  received  the  goal.  The  facilitator  counted  the  number  of  comments  input  by 
each  subject  during  the  first  scenario,  created  the  feedback  graphic  attached  in  Appendix 
D,  and  individually  showed  each  subject  the  results.  The  graphic  given  to  the  subjects 
depicted  three  types  of  information:  the  individual’s  total  comments  during  the  first 
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scenario,  the  total  amount  of  comments  contributed  by  other  group  members,  and  the 
average  number  of  individual  comments  provided  during  pilot  studies  for  the  scenario. 
The  facilitator  explained  each  type  of  information  to  each  subject  individually  and 
continued  the  process  until  all  subjects  had  received  the  information.  Experiment 
participants  were  unaware  if  their  group  was  to  receive  feedback  and  a  goal  or  just  goal 
information. 


Table  2  Reliability  Analysis  -Feedback  Manipulation  Checks 


M 

SD 

Manipulation  Check  1 :  Feedback 

5.18 

1.35 

My  group  received  information  on  how  well  we  shared  information  during  the 

5.45 

1.62 

first  task. 

Each  member  of  my  group  knew  how  much  they  had  contributed  to  the  group 

5.35 

1.62 

during  the  first  task. 

I  knew  how  much  information  other  members  of  my  group  shared  during  the 

4.93 

1.74 

first  task. 

Summary  results  for  the  manipulation  of  feedback  are  provided  in  Figure  1.  The 
manipulation  of  feedback  results  indicate  that  this  manipulation  was  successful. 
Participants  who  were  given  feedback  registered  a  marginal  mean  of  5.76  (s  =  1.03), 
compared  to  a  marginal  mean  of  4.54  (s  =  1.36)  for  those  who  did  not  receive  the 
manipulation.  The  summary  statistics  confirms  that  the  difference  in  means  is 
statistically  reliable,  (F(l,  208)  =  52.16,  p  <  0.001,  eta2  =  0.205). 
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Figure  1  Plot  of  treatment  means  for  manipulation  of  feedback. 


3.5.2  Labeling  and  Anonymity  Manipulation 

The  labeling  and  anonymity  was  successfully  manipulated  by  assigning  subjects 
to  a  condition  where  they  knew  the  label  of  the  comment  provider,  they  did  not  know  the 
label  of  the  comment  provider,  or  they  knew  the  comment  and  user  label  of  the 
individual.  The  subjects  did  not  know  the  other  possible  treatments  that  they  could  be 
assigned  to.  Most  of  the  subjects  had  not  used  a  GSS  before  and  did  not  know  if 
comments  would  be  labeled  or  unlabeled.  One  interesting  item  of  note  is  that  some 
participants  in  the  no  comment  label  treatment  went  ahead  and  identified  themselves  to 
other  members  of  the  group.  After  interviewing  subjects  after  the  session,  it  appeared 
subjects  wanted  to  know  the  identity  of  other  group  members  to  verify  who  submitted 
individual  comments.  The  scores  for  these  subject  groups  were  removed  from  the  data 
and  replaced  with  valid  groups. 
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Table  3  Reliability  Analysis  -  Labeling  Manipulation  Check 


M  SD  a 


Manipulation  Check  2:  Comment  Labeling 

4.64 

1.50  .89 

I  could  tell  if  someone  was  sharing  more  information  than  other  members  of 

4.64 

1.65 

the  group. 

I  could  tell  if  someone  participated  less  than  other  members  of  the  group. 

4.47 

1.70 

Other  group  members  could  judge  the  extent  that  I  participated  in  the  group. 

4.73 

1.62 

Summary  results  for  the  manipulation  of  labeling  are  provided  in  Figure  2.  The 
manipulation  of  comment  labeling  was  also  successful.  Participants  who  were  exposed 
to  labeled  comments  registered  a  marginal  mean  of  5.09  (s  =  1.15).  Those  who  had 
comments  labeled  combined  with  a  placard  registered  a  marginal  mean  of  5.28  (s  =  1.25). 
Those  who  had  no  comment  labeling  registered  a  marginal  mean  of  3.40  (s  =  1.42).  A 
review  of  summary  statistics  confirms  that  the  difference  in  means  is  statistically  reliable, 
(F(2,  208)  =  44.28,  p  <  0.001,  eta2  =  0.305). 


Figure  2  Plot  of  treatment  means  for  manipulation  of  comment  labeling. 
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Table  4  Reliability  Analysis  -  Anonymity  Manipulation  Check 


M  SD  a 


Manipulation  Check  3:  Anonymity 
I  could  recognize  the  originator  of  most  comments. 

Other  group  members  could  connect  me  to  the  comments  I  made. 

Other  group  members  knew  when  I  made  a  contribution  to  the  group. 


4.91  1.50  .87 

4.80  1.84 

4.93  1.62 

5.07  1.57 


Summary  results  for  the  manipulation  of  anonymity  are  provided  in  Figure  3. 
Review  of  the  ANOVA  for  the  final  manipulation  of  anonymity  shows  that  this 
manipulation  was  also  successful.  Participants  who  were  identified  through  labeled 
comments  registered  a  marginal  mean  of  5.02  (s  =  1.34).  Those  who  had  comments 
labeled  along  with  a  placard  registered  a  marginal  mean  of  5.84  (s  =  1.08).  Those  who 
remained  anonymous  registered  a  marginal  mean  of  3.77  (s  =  1.31).  A  review  of 
summary  statistics  confirms  that  the  difference  in  means  is  statistically  reliable,  (F(2, 
208)  =  46.48,  p  <  0.001),  and  tells  us  that  the  effect  of  the  manipulation  on  this  dependent 
variable  was  the  strongest  of  the  three  with  an  eta2  =  0.315. 
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—•—Comments  Labeled  Comments  with  Placard  No  Labels 


Figure  3  Plot  of  treatment  means  for  manipulation  of  anonymity. 


3.6  Task  and  Procedures 

Once  all  four  participants  arrived,  they  were  seated  in  the  break  room  to  begin  the 
initial  portion  of  the  study.  Working  from  a  script  (Appendix  E),  the  facilitator 
introduced  himself  and  his  assistant  who  administered  the  experiment.  The  assistant 
handed  a  manila  folder  to  each  subject  which  contained  a  consent  form,  a  personality 
questionnaire,  and  a  paper  copy  of  the  moon  scenario.  The  participants  were  asked  to 
read  and  sign  a  consent  form  stating  the  subjects’  rights  during  the  study  and  that 
participation  was  voluntarily.  When  subjects  signed  the  consent  form  (Appendix  F),  the 
assistant  collected  them  and  placed  them  in  a  clearly  marked  folder  labeled  “Consent 
Forms”.  The  facilitator  assured  the  subjects  that  any  information  provided  during  the 
experiment  could  not  be  traced  back  to  them. 

Subjects  were  then  asked  to  complete  the  personality  questionnaire  which 
collected  data  for  another  thesis.  Once  this  was  completed,  subjects  placed  the 


42 


questionnaire  in  the  manila  folder.  The  participants  were  then  asked  to  read  the  moon 
scenario  and  using  a  pencil  and  paper  individually  rank  the  fifteen  items  in  order  of 
priority.  Once  subjects  completed  the  task,  they  were  to  place  the  scenario  paper  into 
their  manila  folder.  The  facilitator  then  explained  to  the  subjects  group  decision-making 
and  problem  solving  skills.  Emphasized  during  the  presentation  was  the  need  for 
members  to  freely  generate  ideas  or  to  brainstorm.  It  was  explained  that  once  ideas  are 
exhausted,  the  group  then  must  reach  a  consensus  on  the  order  of  items  in  which  all 
members  can  endorse. 

After  these  activities,  members  were  taken  to  the  task  room,  which  was  where  the 
GSS  lab  was  configured.  After  arriving  in  the  GSS  area,  subjects  were  given  a  short 
demonstration  on  how  to  use  the  GroupSystems  tools  of  categorizor  and  vote. 
Categorizor  is  a  tool  within  GroupSystems  where  participants  can  add  comments  to  the 
GSS  viewable  to  all  group  members.  Vote  allows  participants  to  drag  and  drop  items  in  a 
list,  place  them  in  a  desired  order,  and  submit  their  vote.  Once  all  votes  are  submitted, 
GroupSystems  tabulates  the  ranked  items’  mean,  allowing  the  facilitator  to  then  display 
the  voting  results. 

Groups  were  given  a  short  practice  scenario  where  the  team  had  to  discuss 
possible  ways  to  rank  order  a  list  of  names  which  is  provided  in  Appendix  G.  Groups 
were  given  five  minutes  to  generate  ideas  and  then  the  facilitator  ended  the  discussion 
and  allowed  each  member  to  individually  vote  on  how  the  list  should  be  ranked.  The 
facilitator  waited  until  all  subjects  had  voted  and  then  showed  the  final  results  to  the  team 
on  an  overhead  screen  via  the  in-focus  machine.  The  facilitator  described  the  voting 
results  and  emphasized  the  concept  of  consensus  to  the  group.  Team  members  were  told 


43 


that  in  the  actual  scenario  they  would  be  given  extra  time  to  decide  if  they  were  satisfied 
with  the  voting  results  or  they  could  be  given  the  option  to  discuss  for  an  additional  five 
minutes  and  revote. 

After  the  practice  session,  groups  were  given  fifteen  minutes  to  discuss  the  fifteen 
items  from  the  moon  scenario  via  the  GSS.  Groups  were  told  to  focus  on  discussing  the 
merits  of  items,  not  on  the  rank  order  list  since  at  the  end  of  fifteen  minutes  they  would 
be  allowed  to  rank  order  the  list,  and  submit  their  vote.  The  facilitator  told  the  group  that 
he  would  not  answer  any  questions  concerning  the  scenario,  but  would  assist  with  any 
GSS  specific  questions.  Subjects  were  told  they  would  be  notified  when  they  had  five 
and  two  minutes  remaining  in  the  discussion.  Teams  were  then  shown  the  list  of  items 
and  the  discussion  time  began. 

At  the  end  of  the  fifteen  minutes  the  facilitator  stopped  the  discussion  and  allowed 
each  member  to  rank  order  the  items  based  on  the  groups  discussion.  Once  each  member 
had  ranked  the  items  and  submitted  their  vote,  the  results  were  compiled  and  shown  to 
the  group  via  the  in-focus  machine.  The  group  then  had  to  decide  if  they  were  satisfied 
with  the  results  of  the  vote  or  they  could  be  given  an  additional  five  minutes  for 
discussion  and  then  vote  again.  If  members  decided  they  were  happy  with  the  results 
they  were  given  a  five-minute  break  and  told  to  meet  in  the  break  room.  If  the  team 
wanted  further  discussion,  they  were  given  five  more  minutes  for  additional  discussion 
and  then  re  voted.  The  results  were  then  shown  to  the  group  and  they  were  given  a  five- 
minute  break  and  told  to  regroup  in  the  break  room. 

While  the  group  was  on  break,  the  facilitator  and  his  assistant  counted  the  number 
of  comments  that  had  been  input  by  each  member.  This  was  done  by  going  to  each 
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computer  terminal  and  viewing  the  session  log,  which  saves  the  comments  input  by  each 
subject.  If  feedback  was  required  for  the  group,  the  facilitator  would  input  the 
information  into  a  Microsoft  Excel  97  spreadsheet  and  produced  the  feedback  chart  for 
the  group.  An  example  chart  is  depicted  in  Appendix  D. 

When  the  subjects  returned  from  their  break,  they  were  given  a  five-item  survey 
to  measure  their  commitment  to  the  group  ranking  of  the  moon  scenario.  Once  the 
questionnaire  was  completed,  it  was  placed  into  their  manila  folder. 

After  completing  the  questionnaire,  the  assistant  began  the  goal  discussion  with 
the  group.  He  showed  the  group  two  charts  depicting  group  participation  rates 
(Appendix  H).  One  chart  showed  a  group  that  participates  equally  and  the  other  showed 
unequal  participation  rates.  He  then  discussed  that  groups  that  participate  more  evenly 
will  generally  produce  better  results  since  one  person  does  not  dominate  the  entire 
meeting  and  everyone  is  able  to  voice  their  opinion.  Also  mentioned  was  that  the  more 
comments  input  during  a  meeting  the  greater  the  chance  the  optimum  solution  would  be 
found  in  the  comments.  Subjects  were  then  given  the  goal  for  the  next  task,  which  was 
participate  equally  while  maximizing  your  number  of  comments  on  the  desert  scenario. 
If  the  group  was  part  of  the  feedback  manipulation  they  received  the  following  additional 
information. 

Each  member  in  the  assessment  feedback  condition  was  shown  a  chart  found  in 
Appendix  D.  The  assistant  approached  each  subject,  then  described  and  showed  the 
individuals’  total  comments  during  the  first  scenario,  the  total  amount  of  comments 
contributed  by  other  group  members,  and  the  average  number  of  individual  comments 
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provided  during  pilot  studies  for  the  scenario.  This  concluded  the  feedback  portion  of  the 
experiment. 

After  the  discussion  of  goals  and  feedback,  the  assistant  provided  each  subject 
with  a  paper  copy  of  the  desert  scenario,  which  members  completed  individually.  Once 
the  scenario  was  completed  the  subjects  placed  the  copy  in  their  manila  folder  and  the 
assistant  reiterated  the  goal  statement  again  to  the  group  before  they  proceeded  to  the  task 
room. 

In  the  task  room,  the  subjects  were  given  fifteen  minutes  to  discuss  the  fifteen 
items  from  the  desert  scenario.  Groups  were  told  to  focus  on  discussing  the  merits  of 
items  and  not  on  the  rank  order  list  since  at  the  end  of  fifteen  minutes  they  would  be 
allowed  to  rank  order  the  list  and  submit  their  vote.  Teams  were  then  shown  the  list  of 
items  in  categorizer  and  the  discussion  time  began.  The  same  process  of  discussion  and 
voting  was  repeated  just  as  during  the  moon  scenario. 

Once  the  team  had  submitted  their  final  vote,  they  were  led  back  to  the  break 
room  where  they  were  given  another  five-item  survey  to  measure  their  commitment  to 
the  group  ranking  of  the  desert  scenario.  Once  the  questionnaire  was  completed,  it  was 
placed  into  their  manila  folder. 

Subjects  were  then  given  a  39-item  questionnaire,  attached  in  Appendix  C,  to 
collect  data  for  the  experiment  constructs  and  manipulation  checks.  Once  the 
questionnaire  was  completed,  it  was  placed  into  the  subject’s  manila  folder. 

The  subjects  were  thanked  for  their  time  and  asked  if  they  had  any  questions 
concerning  GSS  or  the  experiment.  Once  all  questions  were  answered,  the  subjects  were 
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allowed  to  leave.  The  entire  script  for  both  parts  of  his  experiment  is  found  in 
Appendix  E. 

While  the  assistant  was  conducting  the  debrief,  the  facilitator  compiled  the 
comments  submitted  during  the  desert  scenario.  At  each  computer  station  he  placed  a 
disk  in  the  machine  to  preserve  a  record  of  the  comments  generated  during  both  sessions. 
To  ensure  an  accurate  history  of  data  was  kept,  the  groups  voting  results  were  saved  on 
the  facilitator  workstation’s  harddrive,  on  diskettes,  and  printed  for  a  hardcopy  format. 
All  material  that  was  placed  in  the  manila  folders  was  labeled  by  session  number  and 
coded  for  analysis  at  a  later  time. 

3.7  Hypothesis  Outcome  Measures 

As  discussed  earlier,  it  is  believed  that  the  individual  and  combined  effects  of 
summary  feedback  from  the  initial  task  and  the  comment  labeling  created  through 
comment/user  labeling  will  effect  individual  user  attitudes,  participant’s  information 
sharing  behavior,  group  decision  quality,  and  unequal  participation  among  members  of 
the  group.  The  definitions  for  all  four  constructs  are  depicted  in  Table  5 
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Table  5  Construct  Definitions _ _ _ _ _ 

Construct  1.  Participant’s  Attitudes 

Definition:  Individual  users’  dispositions  towards  user  satisfaction  with  dynamics 
of  the  participant  group,  user  perception  of  ease  of  use,  user  belief  in  meeting 
utility,  individual’s  ability  to  communicate,  status  effects  of  group,  and 
participation  in  task  related  ideas. 

Construct  2.  Participant’s  Information  Sharing  Behavior 

Definition:  The  average  number  of  intellective  comments  provided  by  each  group 
in  the  desert  scenario. 

Construct  3.  Group  Decision  Quality 

Definition:  Value  of  the  group’s  desert  scenario  score. 

Construct  4.  Unequal  Participation 

Definition:  The  variation  of  the  number  of  intellective  comments  generated 
within  each  group  in  the  desert  scenario. 

The  first  construct  employed  six  scales.  The  scales  measured  user  satisfaction 
with  dynamics  of  the  participant  group,  user  perception  of  ease  of  use,  user  belief  m 
meeting  utility,  individual’s  ability  to  communicate,  status  effects  of  group,  and 
participation  in  task  related  ideas, 

The  second  construct  was  participation.  This  was  quantitatively  analyzed  by 
looking  at  the  number  of  intellective  comments  input  during  the  desert  scenario.  For  the 
purposes  of  this  study,  intellective  comments  were  defined  as  comments  specifically 
related  to  the  intellective  problem-solving  task,  or  the  process  by  which  the  group 
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attempted  to  solve  the  task.  Researchers  coded  intellective  participation  according  to 
individual  thought  processes  within  each  entry.  For  instance,  if  a  participant  typed  a 
paragraph  of  thoughts  prior  to  striking  the  <Enter>  key,  each  separate  thought  was  coded 
as  an  intellective  comment,  rather  than  the  entire  entry  counted  as  a  single  input  Other 
comments  that  simply  indicated  agreement  with  other  members’  intellective  comments, 
affirmed  previously  submitted  intellective  comments,  or  refuted  previously  submitted 
intellective  comments  were  coded  as  affirmation  comments.  The  affirmation  comments 
were  not  included  in  the  count  of  comments  input  during  a  session.  Individual  number  of 
intellective  comments  for  each  participant  in  the  desert  scenario  was  totaled  and  divided 
by  four  to  determine  the  average  number  of  intellective  comments  input  by  the  group. 
The  data  was  then  used  to  determine  if  the  manipulations  had  any  effect  on  the 
participation  rates  during  the  experiment.  In  the  treatments,  one  researcher  coded  all 
comments  to  maintain  consistency.  A  second  researcher  coded  a  random  sample  of 
comments  comprising  15%  of  the  treatment  groups.  Results  found  that  inter-rater 
reliability  using  Cronbach's  alpha  was  assessed  at  .93. 

The  third  construct  group  decision  quality  was  quantitatively  measured  by 
comparing  the  group  scores  from  the  desert  scenario.  Scenario  scores  were  calculated  by 
taking  the  groups  item  rankings  and  comparing  them  to  the  suggested  expert’s  ranking. 
The  expert  ranking  for  the  moon  scenario  comes  from  a  group  of  NASA  experts  (Hall, 
1971:51).  The  desert  scenario  expert  ranking  comes  from  the  former  commander  of  the 
United  States  Air  Force  Desert  Survival  school.  The  absolute  difference  between  item 
placements  created  a  value  for  each  item.  These  were  summed  for  each  item  to  create  a 
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group  decision  score.  If  a  team  correctly  placed  all  items  in  correct  order,  their  team 
score  would  be  zero. 

The  fourth  construct  unequal  participation  was  quantitatively  measured  by 
comparing  the  variation  between  the  number  of  intellective  comments  generated  in  the 
desert  scenario  per  group.  The  smaller  the  variation  for  each  group  equates  to  more  equal 
participation  levels. 

3.8  Survey  Design 

The  user  attitudes  construct  used  six  measured  variables  described  in  Tabic  6 
below.  Each  variable  was  measured  with  five  items  using  a  seven-point  Likert  scale. 
The  final  survey  included  thirty-nine  randomized  items  [(6  measured  variables  *  5 
questions  each)  +  (3  manipulation  checks  *  3  questions  each)] . 

All  survey  data  was  input  into  a  Microsoft  Excel  97  spreadsheet  and  then 
imported  into  the  SPSS  10.0  statistical  software  package  to  determine  Cronbach’s  alpha 
and  to  conduct  the  ANOVA  analysis.  The  requested  output  from  SPSS  was  a  correlation 
matrix,  reliability  coefficient,  mean,  and  standard  deviation  for  each  questionnaire  item. 
Scale  reliability  was  estimated  by  calculating  the  internal  consistency  of  each  multi-item 
scale  as  indexed  by  Cronbach's  coefficient  alpha  (a).  The  mean  for  each  measured 
variable  was  calculated  by  dividing  the  grand  mean  by  the  number  of  items  included  in 
each  measure. 
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Table  6  Reliability  Analysis  -  User  Attitudes 


M 

SD 

a 

User  Satisfaction  With  Dynamics  of  the  Participant  Group 

5.95 

.81 

.86 

I  would  not  mind  working  with  this  group  again. 

6.17 

0.93 

I  am  pleased  with  the  performance  of  our  group. 

5.95 

0.92 

In  my  opinion,  we  worked  effectively  as  a  group. 

5.74 

1.17 

I  found  the  other  group  members  easy  to  work  with. 

5.99 

1.00 

I  enjoyed  participating  in  the  group  activity. 

6.04 

1.02 

User  Belief  in  Meeting  Utility 

5.56 

1.06 

.87 

The  tools  and  processes  helped  us  exchange  information. 

5.76 

1.21 

The  tools  and  processes  helped  us  make  good  use  of  the  information  we  shared. 

5.62 

1.28 

The  tools  and  processes  helped  us  to  know  about  the  things  we  agreed  on. 

5.67 

1.27 

The  tools  and  processes  helped  us  to  focus  on  the  points  where  we  disagreed. 

5.25 

1.47 

The  tools  and  processes  helped  us  to  know  the  extent  we  achieved  consensus. 

5.77 

1.30 

Individual’s  Ability  to  Communicate 

5.89 

.80 

.12 

I  experienced  few  problems  expressing  my  ideas  to  the  other  group  members. 

5.43 

1.53 

I  felt  comfortable  putting  forward  my  own  ideas. 

6.39 

0.88 

I  had  little  trouble  understanding  the  points  made  by  other  group  members. 

5.48 

1.26 

I  was  able  to  comment  on  the  ideas  submitted  by  other  group  members  during  the 

6.04 

1.01 

session. 

I  think  the  other  group  members  received  the  information  I  shared. 

6.02 

1.08 

Table  6  continues 
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Table  6  (Continued) 

User  Perception  of  Ease  of  Use 


Learning  to  use  the  tools  and  process  provided  was  easy  for  me. 

I  found  it  easy  to  use  the  tools  and  process  to  share  information. 

I  found  it  easy  to  use  the  tools  and  process  to  receive  information. 

I  found  it  easy  to  use  the  tools  and  process  to  make  sense  of  shared  information. 

I  found  it  easy  to  use  the  tools  and  process  to  help  my  group  complete  the  task. 

Status  Effects  of  Group 

One  or  more  of  the  group  members  tried  to  intimidate  the  others. 

One  or  more  of  the  group  members  tried  to  force  their  opinions  on  the  group. 

I  felt  inhibited  from  participating  in  the  discussion  because  of  the  behavior  of  one  or 
more  of  the  other  members. 

I  felt  pressure  to  conform  to  a  particular  viewpoint. 

One  or  more  of  the  group  members  tried  to  dominate  the  discussion. 

Participation  in  Task  Related  Ideas 

Everyone  in  the  group  was  very  involved  in  the  group’s  discussion. 

I  got  a  lot  of  good  ideas  about  ranking  from  the  other  members  of  my  group. 
Everyone  in  my  group  seemed  to  contribute  all  of  the  ideas  they  had  about  the  task. 
No  one  seemed  to  be  holding  back  information. 

My  group  shared  a  lot  of  information  while  we  completed  this  task. 


6.06 

.92 

6.58 

0.67 

6.14 

1.22 

6.07 

1.18 

5.93 

1.11 

5.90 

1.26 

2.22 

1.11 

2.23 

1.58 

2.77 

1.71 

1.70 

0.98 

2.23 

1.45 

2.38 

1.43 

5.76 

.83 

5.83 

1.13 

5.78 

1.06 

5.71 

1.13 

5.84 

1.20 

5.93 

0.92 

.89 


.82 


.82 
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3.9  Statistical  Analysis 

A  multivariate  ANOVA  was  performed  on  the  data  collected  during  the 
experiment  which  will  compare  the  means  of  the  populations.  An  ANOVA  uses  data  to 
compare  several  treatments  in  order  to  determine  if  they  achieve  different  results  (Lapin, 
1978:525).  The  ANOVA  was  used  to  determine  if  there  were  statistically  reliable 
differences  among  the  means  due  to  the  feedback,  anonymity,  or  their  interaction. 

The  threshold  for  the  probability  of  a  Type  I  error  (falsely  rejecting  a  null 
hypothesis)  was  set  to  less  than  five  percent.  When  analyzing  the  data,  an  F  statistic  was 
used  to  test  if  we  can  reject  the  null  hypothesis.  If  the  F  statistic  was  sufficiently  large,  so 
that  the  likelihood  of  its  occurring  by  chance  is  less  than  .05,  then  we  reject  the  null 
hypothesis.  If  the  calculated  F-statistic  is  greater  than  the  critical  value  of  F  associated 
with  an  a  <  0.05,  then  the  variation  is  attributed  to  the  difference  between  the  treatments, 
and  the  null  hypothesis  will  be  rejected  in  favor  of  the  alternative  hypothesis  (Lapin, 
1978:533). 

The  eta-squared  statistic  was  used  to  determine  the  effect  size.  Eta-squared  is  the 
proportion  of  the  total  variability  in  the  dependent  variable  accounted  for  by  the  variation 
in  the  independent  variable.  It  is  calculated  as  the  ratio  of  the  effect  variance  (SSeffect)  to 
the  total  variance  (SStotai)-  An  eta-squared  near  0.05  is  generally  considered  a  weak 
effect,  0.10  a  moderate  effect,  and  an  eta2  greater  than  0.15  a  strong  effect.  These 
standards,  were  used  when  analyzing  the  results  from  the  experiment  (Jaccard  and 
Becker,  1997:275-276). 
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3.10  Summary 

The  purpose  of  this  chapter  was  to  describe  the  2x3  factorial  experiment  which 
explored  the  impact  of  coincidental  feedback  and  feedback  and  goal  setting  on  the 
efficacy  of  GSS.  Chapter  Three  described  the  equipment  and  facilities  used,  the  subjects 
used  in  the  study,  and  the  task  and  procedures  of  the  experiment.  Also  discussed  were 
the  questions  used  for  the  survey.  The  chapter  then  explained  and  defined  the  constructs 
of  user  attitudes,  participation,  group  decision  quality,  and  unequal  participation.  Finally, 
the  chapter  presents  the  statistical  methods  that  were  used  to  analyze  and  discover  the 
relationship  between  the  previously  mentioned  independent  variables  and  process 
outcomes. 

Chapter  Four  will  present  the  results  of  the  statistical  analysis  in  narrative  and 
graphical  form. 
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IV.  Analysis  of  Data 


4.1  Introduction 

This  chapter  provides  a  statistical  analysis  and  graphical  display  of  the  data 
collected  during  the  experiment.  Chapter  Five  presents  a  more  detailed  description  of  the 
findings  based  on  the  previously  mentioned  hypothesis. 


4.2  Information  Sharing 

The  following  section  presents  the  data  concerning  information  sharing  behavior. 
The  data  will  be  used  to  answer  hypotheses  1A,  2 A,  3 A,  4 A,  5 A,  and  6A.  The  summary 
results  of  ANOVA  information  sharing  behavior  are  presented  in  Figure  4  and  Table  7. 


Table  7  Means  and  Standard  Deviations  for  Information  Sharing  Behavior 


Comments  Labeled 

No  Labels 

Comments  with  Placard 

Mean 

Std  Dev 

Mean 

Std  Dev 

Std  Dev  * 

Feedback 

21.83 

21.78 

8.22 

No  Feedback 

23.94 

24.97 

6.24 

— 

Figure  4  Plot  of  treatment  means  for  information  sharing  behavior. 
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Providing  different  anonymity  levels  appears  to  have  had  an  effect  on 
participation  (F(2,  52)  =  4.204,  p  =  0.021,  eta2  =  0.155).  Groups  that  had  no  labels 
recorded  a  mean  of  23.38  (s=7.24)  intellective  comments  per  session,  groups  that  had 
comment  labels  recorded  a  mean  of  22.88  (s=9.25),  and  groups  that  received  coincidental 
feedback  had  a  mean  of  16.99  (s=2.91)  intellective  comments  per  session.  ANOVA 
results  show  that  providing  feedback  had  no  effect  on  group  participation  (F(l,  52)  = 
1.138,  p  =  0.292,  eta2  =  0.024).  There  is  also  evidence  against  the  presence  of  an 
interaction  between  the  feedback  and  labeling  (F(2,  52)  =  .094,  p  =  0.910,  eta2  =  0.004). 

4.3  Quality  of  Group  Decision 

The  following  section  presents  the  data  concerning  quality  of  decision  for  the 
group.  The  data  will  be  used  to  answer  hypotheses  IB,  2B,  3B,  4B.  The  summary  results 
of  ANOVA  group  decision  quality  are  presented  in  Figure  5  and  Table  8. 
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Table  8  Means  and  Standard  Deviations  for  Group  Decision  Quality 


Comments  Labeled 

No  Labels 

Comments  with  Placard 

Mean 

Std  Dev 

Mean 

|  Std  Dev 

Mean 

Std  Dev 

Feedback 

58.20 

9.82 

59.75 

|  22.66 

60.89 

10.64 

No  Feedback 

13.82 

59.50 

j  10.24 

56.25 

13.83 

-♦—Comments  Labeled  —a— Comments  with  Placard . & . No  Labels 


Figure  5  Plot  of  treatment  means  for  group  decision  quality. 


A  review  of  ANOVA  results  shows  that  providing  different  anonymity  levels  had 
no  effect  on  the  group  decision  quality  (F(2,  52)  =  .690,  p  =  0.507,  eta2  =  0.029). 
Similarly,  feedback  had  no  effect  on  the  group  decision  quality  (F(l,  52)  =  1.135,  p  = 
0.292,  eta2  =  0.024). 


4.4  Unequal  Participation 

The  following  section  presents  the  data  concerning  unequal  participation.  The 
data  will  be  used  to  answer  hypotheses  1C,  2C,  3C,  4C,  6B,  and  7A.  The  summary 
results  of  ANOVA  unequal  participation  are  presented  in  Figure  6  and  Table  9. 
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Table  9  Means  and  Standard  Deviations  for  Unequal  Participation 


Comments  Labeled 

No  Labels 

Comments  with  Placard 

Mean 

Std  Dev 

Mean 

Std  Dev 

Mean 

Std  Dev 

Feedback 

.15 

.005 

.23 

.126 

.20 

.005 

No  Feedback 

|  .18 

.005 

.20 

|  .107 

.22 

.100 

Figure  6  Plot  of  treatment  means  for  unequal  participation. 


A  review  of  the  ANOVA  results  shows  that  the  different  anonymity  levels  had  no 
effect  on  the  unequal  participation  rates  (F(2,  52)  =  2.043,  p  =  0.141,  eta2  =  0.082). 
Results  also  indicate  that  providing  feedback  had  no  effect  on  unequal  participation  rates 
among  the  group  (F(l,  52)  =  .121,  p  =  0.729,  eta2  =  0.003).  There  is  also  no  evidence  to 
support  an  interaction  effect  between  feedback  and  anonymity  levels  (F(2, 52)  =  .651,  p  = 
0.526,  eta2  =  0.028). 


4.5  Group  Member  Attitudes  toward  the  Group  and  GSS 

The  following  sections  present  the  data  concerning  user  attitudes  with  the  GSS 
meeting.  The  data  will  be  used  to  answer  hypotheses  ID,  2D,  3D,  4D,  and  5B. 
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4.5. 1  User  Satisfaction  With  Dynamics  of  the  Participant  Group 

The  summary  results  of  ANOVA  user  satisfaction  with  dynamics  of  the 
participant  group,  are  presented  in  Figure  7  and  Table  10. 


Table  10  Means  and  Standards  Deviations  for  User  Satisfaction  With  Dynamics  of  the 


Participant  Group 


Comments  Labeled 

No  Labels 

Comments  with  Placard 

Mean 

Std  Dev 

Mean 

Std  Dev 

Mean 

Std  Dev 

Feedback 

1.00 

6.17 

.61 

No  Feedback 

.89 

6.11 

.69 

♦  -  Comments  Labeled-*—  Comments  with  Placard  No  Labels 


7.00 

6.00 

5.00 

4.00 

3.00 

2.00 

1.00 


No  feedback 


Feedback 


Figure  7  Plot  of  treatment  means  for  user  satisfaction  with  dynamics  of  the  participant 
group. 


Providing  different  levels  of  anonymity  had  a  slight  effect  on  user  satisfaction 
with  dynamics  (F(2,  208)  =  3.149,  p  =  0.045,  eta2  =  0.030).  ).  Individuals  that  had 
comments  labeled  with  a  placard  recorded  a  mean  of  6.14  (s=.65)  per  session,  groups 
that  had  comments  labeled  had  a  mean  of  5.80  (s=.95)  and  groups  with  no  labels  had  a 
mean  of  5.93  (s=.76)  per  session.  However,  it  should  be  noted  the  effect  was  not  very 
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strong  since  the  eta2  value  was  low.  A  review  of  the  ANOVA  results  shows  that 
providing  feedback  had  no  effect  on  user  satisfaction  with  dynamics  of  the  participant 
group  (F(l,  208)  =  .078,  p  =  0.780,  eta2  =  0.000).  The  data  also  indicate  there  was  no 
effect  from  the  interaction  of  feedback  and  anonymity  level  (F(2,  208)  =  .512,  p  =  0.600, 
eta2  =  0.005). 


4.5.2  User  Belief  in  Meeting  Utility 

The  summary  results  of  ANOVA  user  belief  in  meeting  utility  are  presented  in 
Figure  8  and  Table  11. 


Table  1 1  Means  and  Standards  Deviations  for  User  Belief  in  Meeting  Utility 


Comments  Labeled 

No  Labels 

Comments  with  Placard 

Mean 

Std  Dev 

Mean 

Std  Dev 

Mean 

Std  Dev 

Feedback 

5.37 

.99 

5.73 

.75 

5.53 

1.20 

No  Feedback  ] 

5.51 

1.14 

5.75 

1.10 

5.57 

1.12 

Figure  8  Plot  of  treatment  means  for  user  belief  in  meeting  utility. 
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A  review  of  the  ANOVA  results  shows  that  different  levels  of  anonymity  had  no 
effect  on  user  belief  in  meeting  utility  (F(2,  208)  =  1.445,  p  =  0.238,  eta2  =  0.014). 
Similarly,  the  use  of  feedback  had  no  effect  on  user  belief  in  meeting  utility  as  compared 
with  groups  in  which  feedback  was  not  received  (F(l,  208)  =  .204,  p  =  0.652,  eta  = 
0.001).  These  main  effects  do  not  show  an  interaction  effect  as  evidenced  by  the 
ANOVA  results  (F(2,  208)  =  .067,  p  =  0.936,  eta2  =  0.001). 


4.5.3  Individual’s  Ability  to  Communicate 

The  summary  results  of  ANOVA  individual’s  ability  to  communicate  are 
presented  in  Figure  9  and  Table  12. 


Table  12  Means  and  Standard  Deviations  for  Individual’s  Ability  to  Communicate 


Comments  Labeled 

No  Labels 

Comments  with  Placard 

Mean 

Std  Dev 

Mean 

Std  Dev 

Mean 

Std  Dev 

Feedback 

5.77 

5.78 

.76 

5.88 

.86 

No  Feedback 

5.83 

mi 

6.04 

.71 

6.08 

.73 

— Comments  Labeled  —m—  Comments  with  Placard  *  No  Labels 


No  feedback  Feedback 


Figure  9  Plot  of  treatment  means  for  individual’s  ability  to  communicate. 
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Providing  different  levels  of  anonymity  had  no  effect  on  an  individual’s  ability  to 
communicate  (F(2,  208)  =  .920,  p  =  0.400,  eta2  =  0.009).  ANOVA  results  demonstrate 
that  providing  feedback  had  no  effect  on  an  individual’s  ability  to  communicate  (F(l, 
208)  =  2.379,  p  =  0.125,  eta2  =  0.012).  Data  also  indicate  there  was  no  effect  from  the 
interaction  of  feedback  and  the  given  anonymity  level  (F(2,  208)  =  .310,  p  =  0.734,  eta2  = 
0.003). 

4.5.4  User  Perception  of  Ease  of  Use 

The  summary  results  of  ANOVA  user  perception  of  ease  of  use  are  presented  in 
Figure  10  and  Table  13. 


Table  13  Means  and  Standard  Deviations  for  User  Perception  of  Ease  of  Use 


Comments  Labeled 

No  Labels 

Comments  with  Placard 

Mean 

Std  Dev 

Mean 

Std  Dev 

Mean 

Std  Dev 

Feedback 

5.96 

.97 

6.09 

.93 

6.11 

.92 

No  Feedback 

6.09 

.93 

5.99 

1.06 

6.16 

.76 

Figure  10  Plot  of  treatment  means  for  user  perception  of  ease  of  use 


A  review  of  the  ANOVA  results  indicates  that  providing  different  levels  of 
anonymity  had  no  effect  (F(2,  208)  =  .257,  p  =  0.774,  eta2  =  0.003).  Also,  providing 
feedback  had  no  effect  on  user  perception  of  ease  of  use  (F(l,  208)  =  .054,  p  =  0.817, 
eta2  =  0.000).  There  is  also  no  evidence  to  support  an  interaction  effect  between  feedback 
and  anonymity  found  in  the  ANOVA  results  (F(2, 208)  =  .289,  p  =  0.749,  eta2  =  0.003). 

4.5.5  Status  Effect  of  Group 

The  summary  results  of  ANOVA  status  effect  of  group  are  presented  in  Figure  1 1 
and  Table  14. 


Table  14  Means  and  Standard  Deviations  for  Status  Effect  of  Group 


Providing  different  levels  of  anonymity  had  no  effect  on  the  status  effect  of  the 
group  (F(2,  208)  =  .284,  p  =  0.753,  eta2  =  0.003).  The  ANOVA  results  also  demonstrate 
that  providing  feedback  had  no  effect  on  the  status  effect  of  the  group  (F(l,  208)  =  .328, 
p  =  0.568,  eta2  =  0.002).  Data  indicate  there  was  no  effects  from  the  interaction  of 
feedback  and  anonymity  level  (F(2,  208)  =  .818,  p  =  0.443,  eta2  =  0.008). 

4.5.6  Participation  in  Task  Related  Ideas 

The  summary  results  of  ANOVA  participation  in  task  related  ideas  are  presented 
in  Figure  12  and  Table  15. 


Table  15  Means  and  Standard  Deviations  for  Participation  in  Task  Related  Ideas 


Comments  Labeled 

No  Labels 

Comments  with  Placard 

Mean 

Std  Dev 

Mean 

Std  Dev 

Mean 

Std  Dev 

Feedback 

5.48 

.85 

5.75 

.82 

5.83 

.79 

No  Feedback 

I  5.82 

.83 

5.87 

.79 

5.90 

.87 

—♦—Comments  Labeled  Comments  with  Placard  »  No  Labels 


No  feedback  Feedback 


Figure  12  Plot  of  treatment  means  for  participation  in  task  related  ideas. 
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Providing  different  levels  of  anonymity  had  no  effect  on  an  individual’s 
participation  in  task  related  ideas  (F(2,  208)  =  1.321,  p  =  0.269,  eta2  =  0.013).  The 
ANOVA  results  demonstrate  that  providing  feedback  had  no  effect  on  an  individual’s 
participation  in  task  related  ideas  (F(l,  208)  =  2.421,  p  =  0.121,  eta2  =  0.012).  Data  also 
indicate  there  was  no  effect  from  the  interaction  of  feedback  and  anonymity  level  (F(2, 
208)  =  .571,  p  =  0.566,  eta2  =  0.006). 

4.6  Summary 

This  chapter  presented  the  ANOVA  results  of  data  collected  during  the 
experiment  through  survey  administration  and  subject  participation.  Chapter  Five 
discusses  the  results  of  the  experiment  by  looking  at  each  of  the  research  hypothesis.  In 
addition  Chapter  Five  summarizes  the  research  findings  and  include  limitations  and 
recommendations  for  future  research. 
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V.  Conclusions  and  Recommendations 


5.1  Introduction 

The  purpose  of  this  study  was  to  look  at  several  competing  GSS  factors  that  could 
help  explain  the  mixed  results  found  in  GSS  research  and  help  facilitators  determine  the 
optimal  configuration  for  GSS  meetings.  The  study  looked  at  different  theories  discussed 
in  Chapter  Two  to  determine  how  they  affect  participation,  decision  quality,  unequal 
participation,  and  member  attitudes  toward  the  GSS.  To  investigate  the  different 
hypothesis,  an  experiment  was  created  that  would  allow  the  competing  factors  to  be 
examined  in  one  experiment.  The  main  and  interactive  effects  could  then  be  studied  to 
determine  which  theory  or  theories  support  the  optimal  GSS  configuration.  The  results  of 
the  experiment  are  described  in  this  final  chapter  by  examining  each  hypothesis  that  was 
stated  in  Chapter  Two.  Finally,  this  chapter  will  present  the  overall  conclusions  of  this 
research  along  with  any  study  limitations  and  recommendations  for  future  research. 

5.2  Hypothesis  1:  Effects  Through  Anonymity  vs  Hypothesis  2:  Effects  Through 
Identification 

Hypothesis  1  and  2  will  be  explained  in  the  same  section  since  each  predicted 
opposite  results  would  occur.  Therefore,  if  the  results  are  statistically  reliable,  the  results 
must  favor  one  or  the  other.  Hypothesis  1  proposed  that  providing  users  anonymity  in  a 
GSS  meeting  would  increase  information  sharing  behavior,  increase  the  quality  of  the 
group  decision,  discourage  unequal  participation,  and  encourage  positive  user  attitudes 
toward  the  GSS  meeting.  Hypothesis  2  stated  that  identifying  users  in  a  GSS  meeting 
would  increase  information  sharing  behavior,  increase  the  quality  of  the  group  decision. 
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discourage  unequal  participation,  and  encourage  positive  user  attitudes  toward  the  GSS 
meeting.  For  ease  of  understanding,  Hypothesis  1  and  2  are  broken  into  four  separate 
sub-hypotheses. 

5.2.1  Hypothesis  1A  and  2A: 

Hypothesis  1A  suggested  that  anonymity  will  have  an  effect  increasing  user 
information  sharing  behavior.  Hypothesis  2A  suggested  that  identification  will  have  an 
effect  increasing  information  sharing  behavior.  Examination  of  the  ANOVA  results  from 
Chapter  Four  presents  strong  evidence  to  support  the  anonymity  factor.  As  1A 
hypothesized,  the  anonymity  condition  produced  more  intellective  comments  than  when 
members  were  identified.  Members  who  remained  anonymous  must  feel  that  anonymity 
allows  them  to  freely  generate  more  ideas  than  when  they  can  be  completely  identified. 

5.2.2  Hypothesis  IB  and  2B: 

Hypothesis  IB  suggests  that  anonymity  will  increase  the  quality  of  decision  for 
the  group.  Hypothesis  posited  that  identification  will  increase  the  quality  of  decision  for 
the  group.  Analysis  of  data  collected  however  presented  no  support  for  either  statement. 

5.2.3  Hypothesis  1C  and  2C: 

Hypothesis  1C  posited  that  anonymity  will  discourage  unequal  participation  from 
group  members.  Hypothesis  2C  suggests  that  identification  will  discourage  unequal 
participation  from  group  members.  Evidence  provided  by  ANOVA,  however,  showed  no 
support  for  either  hypothesis. 
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5.2.4  Hypothesis  ID  and  2D: 

Hypothesis  ID  presented  the  idea  that  anonymity  will  encourage  positive  user 
attitudes  with  the  GSS  meeting.  Hypothesis  2D  presented  the  idea  that  identification  will 
encourage  positive  user  attitudes  with  the  GSS  meeting.  Recalling  from  Chapter  Two, 
user  attitudes  are  measured  from  six  discrete  variables  which  are  user  satisfaction  with 
dynamics  of  the  participant  group,  user  perception  of  ease  of  use,  user  belief  in  meeting 
utility,  individual’s  ability  to  communicate,  status  effects  of  group,  and  participation  in 
task  related  ideas.  Of  the  six  variables,  only  user  satisfaction  with  dynamics  of  the 
participant  group  indicated  any  statistical  difference.  The  difference  was  between  the 
anonymous  condition  and  the  coincidental  feedback  condition.  The  data  indicate  that 
anonymity  did  not  encourage  optimal  user  satisfaction  with  dynamics  of  the  participant 
group.  Group  members  may  feel  that  not  knowing  who  contributed  certain  information 
may  lead  to  dissatisfaction  with  the  meeting. 

5.3  Hypothesis  3:  Effects  Through  Coincidental  Feedback 

Hypothesis  3  proposed  that  providing  users  coincidental  feedback  during  a  GSS 
meeting  would  increase  information  sharing  behavior,  increase  the  quality  of  the  group 
decision,  discourage  unequal  participation,  and  encourage  positive  user  attitudes  toward 
the  GSS  meeting.  Hypothesis  3  is  broken  into  four  separate  sub-hypotheses  discussed 
below. 
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5.3.1  Hypothesis  3 A: 

Hypothesis  3A  posited  that  coincidental  feedback  will  have  an  effect  increasing 
user  information  sharing  behavior.  However,  a  review  of  ANOVA  results  show  no 
support  for  this  statement.  In  fact  results  indicate  that  coincidental  feedback  discourage 
user  information  sharing  behavior.  Participants  generated  the  least  number  of  comments 
under  this  condition. 

5.3.2  Hypothesis  3B: 

Hypothesis  3B  suggests  that  coincidental  feedback  will  increase  the  quality  of 
decision  for  the  group.  Analysis  of  data  collected  from  the  experiment  presented  no 
support  for  this  hypothesis. 

5.3.3  Hypothesis  3C: 

Hypothesis  3C  presented  the  idea  that  coincidental  feedback  will  discourage 
unequal  participation  from  group  members.  Data  analysis  however  shows  no  support  for 
this  statement. 

5.3.4  Hypothesis  3D: 

This  hypothesis  suggests  that  coincidental  feedback  will  encourage  positive  user 
attitudes  with  the  GSS  meeting.  Similar  to  Hypothesis  ID,  this  hypothesis  used  the  same 
six  discrete  variables  to  measure  user  attitudes.  Review  of  ANOVA  results  show  that  the 
only  statistically  reliable  effect  was  user  satisfaction  with  dynamics  of  the  participant 
group.  Members  in  the  comments  labeled  with  a  placard  treatment  scored  the  highest 
among  all  experimental  treatments.  These  group  members  may  have  enjoyed  the  group 
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activity  more  since  they  were  aware  of  who  contributed  comments,  which  supports  the 
coincidental  feedback  theory. 

5.4  Hypothesis  4:  Effects  Through  Goal  Setting  and  Feedback 

Hypothesis  4  proposed  that  a  facilitator  providing  goals  and  feedback  to  users  in  a 
GSS  meeting  would  increase  information  sharing  behavior,  increase  the  quality  of  the 
group  decision,  discourage  unequal  participation,  and  encourage  positive  user  attitudes 
toward  the  GSS  meeting.  This  idea  is  broken  down  into  four  separate  sub-hypotheses 
described  below. 

5.4.1  Hypothesis  4A: 

Hypothesis  4A  presented  the  idea  that  facilitator  provided  goal  setting  and 
feedback  will  have  an  effect  increasing  user  information  sharing  behavior.  Evidence 
provided  by  ANOVA,  however,  showed  no  support  for  this  statement. 

5.4.2  Hypothesis  4B: 

Hypothesis  4B  suggests  that  facilitator  provided  goal  setting  and  feedback  will 
increase  the  quality  of  decision  for  the  group.  This  hypothesis  is  not  supported  from 
ANOVA  evidence. 
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5.4.3  Hypothesis  4C: 

Hypothesis  4C  posited  that  facilitator  provided  goal  setting  and  feedback  will 
discourage  unequal  participation  from  group  members.  Review  of  ANOVA  results 
indicate  no  support  for  this  statement. 

5.4.4  Hypothesis  4D: 

This  hypothesis  suggests  that  facilitator  provided  goal  setting  and  feedback  will 
encourage  positive  user  attitudes  with  the  GSS  meeting.  Similar  to  Hypothesis  ID,  this 
hypothesis  used  the  same  six  discrete  variables  to  measure  user  attitudes.  ANOVA 
results  indicate  that  no  support  for  this  statement. 

5.5  Hypothesis  5:  Effects  Through  Anonymity  and  Goal  Setting  Feedback  Interaction 

Hypothesis  5  stated  that  providing  users  anonymity  in  a  GSS  meeting  along  with 
goal  setting  and  feedback  would  have  an  interactive  effect  that  would  discourage 
information  sharing  behavior  and  encourage  negative  user  attitudes  toward  the  GSS 
meeting.  This  hypothesis  is  broken  into  two  separate  sub-hypotheses. 

5.5.1  Hypothesis  5A: 

Hypothesis  5A  suggests  that  anonymity  and  facilitator  provided  goal  setting  and 
feedback  will  have  an  interactive  effect  discouraging  user  information  sharing  behavior. 
Analysis  of  data  collected  does  not  show  support  for  this  statement. 
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5.5.2  Hypothesis  5B: 

Hypothesis  5B  posited  that  anonymity  and  facilitator  provided  goal  setting  and 
feedback  will  have  an  interactive  effect  encouraging  negative  user  attitudes  with  the  GSS 
meeting.  Review  of  ANOVA  results  indicate  that  an  interactive  effect  was  not  evident. 

5.6  Hypothesis  6:  Effects  Through  Identification  and  Goal  Setting/Feedback  Interaction 
Hypothesis  6  stated  that  identifying  users  in  a  GSS  meeting  and  providing  goal 
setting  and  feedback  would  have  an  interactive  effect  that  would  increase  information 
sharing  behavior  and  discourage  unequal  participation.  For  ease  of  understanding, 
Hypothesis  6  is  broken  into  two  separate  sub-hypotheses  discussed  below. 

5.6.1  Hypothesis  6A: 

Hypothesis  6A  posited  that  identification  and  facilitator  provided  goal  setting  and 
feedback  will  have  an  interactive  effect  increasing  user  information  sharing  behavior. 
Analysis  of  data  collected  during  the  experiment,  however,  presented  no  support  for  this 
hypothesis. 

5.6.2  Hypothesis  6B: 

Hypothesis  6B  posited  that  anonymity  and  facilitator  provided  goal  setting  and 
feedback  will  have  an  interactive  effect  discouraging  unequal  participation  from  group 
members.  Review  of  ANOVA  results  show  no  support  for  this  statement. 
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5.7  Hypothesis  7:  Effects  Through  Coincidental  Feedback  and  Goal  Setting/Feedback 
Interaction 

Hypothesis  7  proposed  that  groups  who  do  not  receive  assessment  feedback  will 
continue  to  show  a  difference  based  on  the  presence  of  coincidental  feedback  through 
comment  labels.  This  hypothesis  is  discussed  below. 

5.7.1  Hypothesis  7  A: 

Review  of  ANOVA  results  for  Hypothesis  7A  show  that  the  interactive  effect  was 
not  evident. 

5.8  Conclusions  and  Recommendations 

This  study  began  by  looking  at  different  structural  and  social  contingencies  that 
would  help  find  the  optimal  configuration  and  administration  of  a  GSS  session.  After 
analyzing  the  collected  survey  and  observational  data,  only  the  theories  of  anonymity  and 
coincidental  feedback  are  partially  supported.  The  experiment  helped  support  the 
anonymity  theory’s  belief  that  under  anonymous  conditions  users  would  generate  more 
intellective  comments.  This  similar  finding  supporting  the  benefits  of  anonymity  has 
been  found  in  other  GSS  studies  and  can  be  added  to  the  GSS  body  of  knowledge.  Only 
one  variable  to  measure  user  attitudes  was  found  to  support  the  coincidental  feedback 
theory.  This  variable  was  the  satisfaction  with  dynamics  of  the  participant  group.  This 
found  that  users  were  more  satisfied  with  the  group  when  they  received  coincidental 
feedback  when  compared  with  being  anonymous.  This  finding  can  also  be  added  to  the 
GSS  body  of  knowledge.  Besides  these  two  findings,  no  evidence  was  available  to 
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support  the  other  hypothesis  listed  in  this  study.  While  the  study  found  some  results  to 
help  optimally  configure  a  GSS  session,  much  work  must  still  be  done  to  discover  the 
optimal  configuration  of  group  support  systems. 

The  Air  Force  is  currently  using  group  support  systems  to  solve  various  problems 
throughout  the  Air  Force.  Prior  research  has  shown  the  importance  of  getting  users  to 
submit  as  many  ideas  as  possibly  to  arrive  at  an  optimal  solution.  This  study  has 
demonstrated  that  anonymous  conditions  in  a  GSS  meeting  provided  more  intellective 
comments  than  GSS  meetings  where  members  can  be  identified.  Administrators  of  the 
group  support  systems  DOME  and  RAPTR  should  use  these  findings  and  continue  to 
leave  group  members  anonymous.  Looking  at  the  data,  all  GSS  conditions  scored  high 
on  satisfaction  with  the  system,  ease  of  use,  ability  to  communicate,  and  meeting  utility. 
These  results  indicate  that  users  are  willing  and  able  to  use  the  tool  to  solve  problems. 
The  Air  Force  should  continue  to  use  group  support  systems  since  they  have  the  potential 
to  outperform  traditional  meetings. 

5.9  Limitations  and  Recommendations  for  Future  Research 

A  limitation  to  this  study  was  the  amount  of  time  and  exposure  the  subjects  were 
given  on  the  GSS.  Even  though  the  experiment  lasted  approximately  two  hours,  the 
features  of  the  GSS  had  to  be  scaled  down  to  allow  the  groups  to  complete  the  tasks  in  a 
reasonable  amount  of  time.  Ideally,  groups  would  be  introduced  to  more  features  of  the 
GSS,  but  this  would  warrant  a  longer  experiment. 

A  second  limitation  to  the  study  was  group  size.  While  this  experiment  used  a 
group  size  of  four,  which  is  common  among  GSS  experiments,  a  larger  group  may  be 
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necessary  to  study  the  effects  of  anonymity  and  identification.  For  example,  in  a  small 
group  of  four  people,  users  may  still  be  able  to  identify  each  other  even  in  an  anonymous 
condition.  If  the  group  was  much  larger,  users  would  have  a  harder  time  identifying 
other  members.  Some  real  world  group  support  systems  have  dozens  of  participants  in  a 
GSS  meeting  and  larger  group  sizes  would  provide  more  external  validity. 

A  third  limitation  was  external  validity  of  the  experiment.  While  the  tasks  given 
to  the  participants  to  solve  in  the  study  are  commonly  used  problem  solving  tasks,  they 
still  do  not  reflect  real  world  problems  that  would  be  solved  in  a  GSS  session.  A  more 
realistic  and  beneficial  experiment  would  be  to  observe  real  world  GSS  meetings  and 
determine  if  findings  in  the  laboratory  can  be  carried  over  to  the  workplace.  This  is  a 
potential  area  for  future  research  since  most  of  the  research  conduct  on  group  support 
systems  has  been  performed  in  the  laboratory,  few  studies  have  been  conducted  outside 
the  laboratory  and  no  studies  have  been  performed  in  an  Air  Force  work  environment.  In 
particular  the  Air  Force  could  look  into  performing  similar  “GSS  introduction”  tasks  used 
in  this  experiment  to  allow  users  to  become  familiar  with  the  system  before  they  begin 
solving  a  real-world  problem.  Taking  a  short  amount  of  time  to  perform  a  sample  GSS 
session  may  save  time  in  the  long  run  since  users  may  become  more  familiar  with  the 
system  and  more  comfortable  performing  tasks  on  the  GSS. 

Another  area  for  future  research  is  that  groups  in  this  experiment  were  either 
given  a  goal  and  no  facilitator  provided  feedback  or  a  goal  with  facilitator  provided 
feedback,  either  way  all  conditions  received  a  goal.  Another  study  that  would  create  a 
condition  where  groups  were  not  given  a  goal  might  be  beneficial  to  look  into.  For 
instance,  a  goal  may  be  providing  enough  information  to  the  users  to  motivate  them  to 
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perform  differently.  This  type  of  study  would  help  determine  whether  or  not  goals 
provide  valuable  information  to  the  GSS  users. 
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Appendix  A:  Moon  Scenario 


You  are  a  member  of  a  space  crew  originally  scheduled  to  rendezvous  with  a 
mother  ship  on  the  lighted  surface  of  the  moon.  Due  to  mechanical  difficulties,  however, 
your  ship  was  forced  to  land  at  a  spot  some  200  miles  from  the  rendezvous  point.  During 
re-entry  and  landing,  much  of  the  equipment  aboard  was  damaged  and,  since  survival 
depends  on  reaching  the  mother  ship,  the  most  critical  items  available  must  be  chosen  for 
the  200-mile  trip. 

The  15  items  left  intact  and  undamaged  after  landing  are  listed  below.  Your  task 
is  to  rank  them  in  terms  of  their  necessity  to  your  crew  in  reaching  the  rendezvous  point. 
Place  the  number  1  by  the  most  crucial  item,  the  number  2  by  the  second  most  crucial, 
and  so  on  through  number  15,  the  least  important. 

_  Box  of  matches 

_  First-aid  kit  containing  injection  needles 

_  Five  gallons  water 

_  Food  concentrate 

_  Life  raft 

_  Magnetic  compass 

_  One  case  dehydrated  milk 

_  Parachute  silk 

_  Portable  heating  unit 

_  Signal  flares 

_  Solar-powered  FM  receiver  transmitter 

_  Stellar  map  (of  the  moon's  constellation) 

_  Two  .45-caliber  pistols 

_ Two  100-pound  tanks  of  oxygen 

_  50  ft.  of  nylon  rope 
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Appendix  B:  Desert  Scenario 

It  is  approximately  10:00  AM  in  mid  August  and  you  have  just  crash-landed  in 

the  Sonora  Desert  in  southwestern  United  States.  The  twin  engine  plane,  containing  the 

bodies  of  the  pilot  and  the  co-pilot,  has  completely  burned.  Only  the  airframe  remains. 

None  of  the  rest  of  you  have  been  injured.  The  pilot  was  unable  to  notify  anyone  of  your 

position  before  the  crash.  However,  he  had  indicated  before  impact  that  you  were  70 

miles  south  -  southwest  from  a  mining  camp  which  is  the  nearest  known  habitation  and 

that  you  were  approximately  65  miles  off  the  course  that  was  filed  in  your  Flight  Plan. 

Before  the  plane  caught  fire  your  Patrol  was  able  to  salvage  the  15  items  listed  on 

the  attached  sheet.  Your  task  is  to  rank  these  items  according  to  their  importance  to  your 

survival.  Place  the  number  1  by  the  most  crucial  item,  the  number  2  by  the  second  most 

cmcial,  and  so  on  through  number  15,  the  least  important. 

_  A  pair  of  sunglasses  per  person 

_  Book  entitled  “Edible  Animals  of  the  Desert” 

_  Bottle  of  salt  tablets  (1000  tablets) 

_ Compress  kit  and  gauze 

_  Cosmetic  Mirror 

_  Flashlight 

_  Magnetic  compass 

_  One  liter  of  water  per  person 

_  One  top  coat  per  person 

_ Parachute  (red  and  white) 

_ Penknife 

_  Plastic  Raincoat  (large  size) 

_  Sectional  Air  Map  of  the  Area 

_  2  liters  of  100%  proof  vodka 

_  .45  caliber  pistol 
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Appendix  C:  Post-Test  Questionnaire 


Answer  the  questions  using  the  following  scale 

1  Strongly  Disagree 

2  Disagree 

3  Disagree  Somewhat 

4  Neither  Agree  Nor  Disagree 

5  Agree  Somewhat 

6  Agree 

7  Strongly  Agree 

_ I  would  not  mind  working  with  this  group  again. 

_ I  am  pleased  with  the  performance  of  our  group. 

_ In  my  opinion,  we  worked  effectively  as  a  group. 

_  I  found  the  other  group  members  easy  to  work  with. 

_  I  enjoyed  participating  in  the  group  activity. 

_  Learning  to  use  the  tools  and  process  provided  was  easy  for  me. 

_  I  found  it  easy  to  use  the  tools  and  process  to  share  information. 

_  I  found  it  easy  to  use  the  tools  and  process  to  receive  information. 

_  I  found  it  easy  to  use  the  tools  and  process  to  make  sense  of  shared  information. 

_  I  found  it  easy  to  use  the  tools  and  process  to  help  my  group  complete  the  task. 

_  The  tools  and  processes  helped  us  exchange  information. 

_ The  tools  and  processes  helped  us  make  good  use  of  the  information  we  shared. 

_  The  tools  and  processes  helped  us  to  know  about  the  things  we  agreed  on. 

_ The  tools  and  processes  helped  us  to  focus  on  the  points  where  we  disagreed. 

_ The  tools  and  processes  helped  us  to  know  the  extent  we  achieved  consensus. 
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Answer  the  questions  using  the  following  scale 


1  Strongly  Disagree 

2  Disagree 

3  Disagree  Somewhat 

4  Neither  Agree  Nor  Disagree 

5  Agree  Somewhat 

6  Agree 

7  Strongly  Agree 

_  I  experienced  few  problems  expressing  my  ideas  to  the  other  group  members. 

_  I  felt  comfortable  putting  forward  my  own  ideas. 

_  I  had  little  trouble  understanding  the  points  made  by  other  group  members. 

_  I  was  able  to  comment  on  the  ideas  submitted  by  other  group  members  during  the 

session. 

_  I  think  the  other  group  members  received  the  information  I  shared. 

_  One  or  more  of  the  group  members  tried  to  intimidate  the  others. 

_  One  or  more  of  the  group  members  tried  to  force  their  opinions  on  the  group. 

_  I  felt  inhibited  from  participating  in  the  discussion  because  of  the  behavior  of  one 

or  more  of  the  other  members. 

_  I  felt  pressure  to  conform  to  a  particular  viewpoint. 

_  One  or  more  of  the  group  members  tried  to  dominate  the  discussion. 

_ Everyone  in  the  group  was  very  involved  in  the  group’s  discussion. 

_  I  got  a  lot  of  good  ideas  about  ranking  from  the  other  members  of  my  group. 

_  Everyone  in  my  group  seemed  to  contribute  all  of  the  ideas  they  had  about  the 

task. 

_  No  one  seemed  to  be  holding  back  information. 
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Answer  the  questions  using  the  following  scale 

1  Strongly  Disagree 

2  Disagree 

3  Disagree  Somewhat 

4  Neither  Agree  Nor  Disagree 

5  Agree  Somewhat 

6  Agree 

7  Strongly  Agree 

_ My  group  shared  a  lot  of  information  while  we  completed  this  task. 

_  My  group  received  information  on  how  well  we  shared  information  during  the 

first  task. 

_  Each  member  of  my  group  knew  how  much  they  had  contributed  to  the  group 

during  the  first  task. 

_  I  knew  how  much  information  other  members  of  my  group  shared  during  the  first 

task. 

_  I  could  recognize  the  originator  of  most  comments. 

_  Other  group  members  could  connect  me  to  the  comments  I  made. 

_  Other  group  members  knew  when  I  made  a  contribution  to  the  group. 

_ I  could  tell  if  someone  was  sharing  more  information  than  other  members  of  the 

group. 

_ I  could  tell  if  someone  participated  less  than  other  members  of  the  group. 

_  Other  group  members  could  judge  the  extent  that  I  participated  in  the  group. 
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Appendix  D:  Feedback  Chart  Example 


Participation  Rate 


Average  #  of 
Comments  on  the  task 
is  23 


□  Your  #  of  Comments 


10 


20 


30 


#  of  Comments 


Appendix  E:  Experiment  Script 


GSS:  Pre-Experiment  Steps 


1.  Ensure  following  items  are  available: 

>  Big  folder  labeled  consent  forms 

>  4  Manila  folders 

>  Attached  via  paper  clip  are 

>  Consent  form 

>  Demographic/Personality  Questionnaire 

>  1  copy  of  Moon  Scenario 

2.  Check  out  Projector  and  printer  with  paper 

3.  In  Group  System  Admin,  click  on  Clear,  then  open  roster,  edit  user  terminal,  set  to 


full-access  user 


4.  Start  Group  Systems  WGE  at  Facilitator  station  and  all  user  stations 
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Preferences 


„J  Prompt  for  Clipboard  Sharing 
✓J  Enable  Automatic  Logging 
_J  Show  Main  Tool  Bar  : 
j|§|  : Prompt  on  Exit 
lljllJse  Large  Font 


8- 


8.  Copy  all  activities  for  session  from: 

For  a  labeled  session:  GSS  Labeled  Study  -  clean  copy 
For  an  unlabeled  session:  GSS  Unlabeled  Study  -  clean  copy 


9.  Paste  to  GSS  Study  -  Current 


GroupSystems  -  GSS  Labeled  Study  -  clean  copy 


: 


LSfi 


g  fcl  I  : 


File  Fc|der$  Options  Window  Help 


0  First  Folder- facilitator 

AFRL  -  CIO  Study  IMGT  699  Spring  2000 
^DemoAFRUCDO 


SS  Study  -  Current 

SS  Uniabeied  Study-  clean  copy 

LSI  Collaboration  Demo  January  1 9, 2000 


85 


10.  Configure  each  GSS  station  for  EACH  ACTIVITY  on  facilitator  station 
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Group  Settings 


j*  All  users  (default) 


Group  Settings 


All  users  (default) 


Ej  View  Comments 
i f\  Add  Comments  : 

!  Modify  Comments 
..  I  View  Participation  Meter 


>  Under  Group  -  Group  Settings  the  following  boxes  must  be  checked 
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11.  Researchers  Label  Subjects  Monitors  with  placard  (if  applicable) 

>  Ensure  four  placards  (blue,  green,  red  and  yellow)  are  available 
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Prep  Room:  Introduction 

1.  When  subjects  arrive,  introduce  yourself.  Have  subjects  wait  in  the  prep  room. 
Tell  subjects  “The  task  will  begin  when  all  participants  have  arrived.” 

2.  Once  all  participants  have  arrived,  have  all  subjects  sit  down  in  the  prep  room. 

3.  Facilitator  says:  “Welcome  to  the  study.  I’m  XX  and  this  is  XX.  We  are  AFIT 
students  conducting  an  experiment  for  our  Masters  degree.  We  will  be  asking  you 
some  questions  about  yourself.  Our  study  looks  at  how  different  types  of  groups 
interact  to  solve  a  problem.  During  the  course  of  this  experiment  you  will  be 
asked  to  complete  three  questionnaires,  receive  some  group  interaction  training, 
and  conduct  tasks  individually  and  as  a  group.  About  half  way  through  this  two 
hour  experiment  you  will  be  given  a  short  break.” 

4.  Facilitator  says:  “My  assistant  will  now  hand  out  a  manila  folder  with  some 
attached  information.  Please  don’t  look  at  the  attached  information  until  asked.” 

5.  Assistant  provides  participants  with  manila  folder. 

6.  Facilitator  says:  “To  begin,  please  remove  the  consent-form  from  the  manila 
folder.  This  form  indicates  your  rights  as  a  participant  in  the  study.  Please  read 
the  consent  form  and  print  and  sign  your  name  at  the  bottom  of  the  page.  Your 
participation  is  voluntary.  If  at  any  time  you  want  to  stop  please  let  the  facilitator 
know.” 

7.  Subjects  read  and  sign  (if  applicable)  Consent  Form 

8.  Assistant  collects  consent  forms 

9.  Facilitator  says,  “This  is  the  only  place  your  name  will  be  recorded  during  this 
experiment.” 

10.  Assistant  puts  consent  forms  in  big  folder  labeled  consent  forms. 

11.  Facilitator  says:  “We  would  now  like  you  to  fill  out  the  individual  characteristics 
questionnaire  attached  to  the  manila  folder.  All  responses  to  this  questionnaire  are 
completely  confidential  and  will  not  be  associated  with  you  as  an  individual.  Use 
the  rating  scale  provided  to  indicate  how  accurately  each  statement  describes  you. 
Think  about  yourself  as  you  generally  are  now  and  not  as  you  wish  to  be  in  the 
future.  Please  read  each  statement  carefully.  Does  anyone  have  any  questions?” 

12.  Subjects  complete  questionnaire. 

13.  Facilitator  says:  “Please  place  the  questionnaire  in  the  manila  folder.” 
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14.  Facilitator  says:  “Now  lets  complete  a  problem  solving  task  individually.  Please 
read  the  scenario  and  complete  the  exercise.  It  will  take  you  approximately  5 
minutes  to  complete  the  exercise.  If  you  finish  early,  please  remain  quiet  until 
everyone  completes  the  exercise.  Please  remove  the  scenario  from  your  manila 
folder.” 

15.  Facilitator  says:  “Please  begin.” 

16.  Subjects  individually  complete  Moon  Scenario 

17.  Facilitator  says:  “Please  place  the  Moon  Scenario  in  the  manila  folder.” 

18.  Facilitator  says:  “Before  we  move  to  another  room  let’s  discuss  group  decision 
making  and  problem  solving  in  general.  The  first  step  is  for  the  group  to  discuss 
the  problem  and  all  pertinent  issues  related  to  the  problem.  One  method  often  used 
to  do  this  is  “brainstorming”  during  which  ideas  are  freely  generated  and  not 
judged  on  quality  or  feasibility.  Once  the  brainstorming  session  is  complete,  the 
group  then  attempts  to  reach  consensus  on  a  solution.  This  does  not  necessarily 
mean  all  individuals  completely  agree  with  the  groups’  decision,  but  the  decision 
is  one  that  all  can  endorse.  There  are  different  methods  groups  use  to  reach 
consensus,  one  of  which  is  voting.  If  the  results  of  the  group  vote  indicate 
agreement,  then  consensus  is  reached.  If  the  group  does  not  have  agreement, 
further  discussion  may  be  required  to  reach  consensus.  Remember  the  purpose  of 
this  study  is  to  look  at  how  different  types  of  groups  interact  to  solve  a  problem. 
Does  anyone  have  any  questions  before  we  move  to  the  next  phase  of  the  study.” 

19.  Facilitator  says:  “Please  pick  up  your  manila  folder  and  follow  me.” 

20.  Assistant  moves  subjects  to  Task  Room  (GSS  Room) 
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GSS  Room:  Training 

1.  Assistant  says:  “Please  take  a  seat  at  one  of  the  computers.” 

2.  Subjects  sit  at  one  of  the  GSS  stations 

3.  Training  Script 

>  Facilitator  flips  UP  projector 

>  As  you  introduce  options  in  GSS  point  to  them  on  the  screen. 

>  Facilitator  says:  “A  group  support  system  is  made  up  of  software,  computers 
and  a  facilitator.  Each  of  your  computers  has  Group  System  software  (point  to 
screen)  loaded  on  it.  This  software  and  hardware  is  often  used  in  the  Air  Force 
to  increase  the  effectiveness  of  decision-making  groups.” 

>  Facilitator  says:  “We  will  only  be  introducing  you  to  a  small  set  of  the 
capabilities  of  a  GSS  because  of  our  limited  time.  As  you  use  this  software, 
please  only  use  the  capabilities  we  introduce  to  you  so  we  can  minimize  the 
impact  on  your  time  and  ours.  For  the  purposes  of  this  study  we  will  be  using 
two  GSS  tools:  Categorizer  and  Vote.” 

>  PAUSE 

>  Facilitator  says:  “Before  we  begin  the  actual  group  problem-solving  scenario, 
we  will  first  guide  you  through  a  brief  training  session.  You  will  be  introduced 
and  allowed  to  practice  with  GSS  Categorizer  and  Vote  tools.  Let’s  begin.” 
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>  Facilitator  starts  participants  in  Training  (Categorizer) 


1  GroupSystems  -  GSS  Study  •*  Curient  ■  [Training  (CategoiizerlJ 

mmmi 

10  Pokier*  £dk  £aiegoriz«  Group  fipbom  \v 

l .  Agenda  People  ^Whiteboard  ^Handout 

5  Opr*on  I^Pepofts  ^Snefcase  log  #$  | 

x)  [UPoKfer^ 

i?  ^  ©  Q  to  0 

•  q  @  t?  %■  %  o  ® 

;i.  t- 

1  First  Folder- facilitator  jr 

<&  AFRL-  CIO  study  IMGT 699  Spring  2000  l: 

‘  ^  Demo  AFRL/CDO  j\ 

|  <@>  GSS  Labeled  Study--  clean  copy 

l  M 

JBBM  mMH 

2.  Susan  Peterson  -  Marketing 

3.  William  Elliott  -  Accounting 

£  GSS  Unlabeled  Study-  clean  copy 

1  <&  VLSI  Collaboration  Demo  January  1 9,  200DV 

1  1 

1  S 

D/fl 

liiiipjyii 

4.  Albert  Smith  ~  Marketing 

0/0 

5.  Michelle  Zunga  -  Accounting 

1 

— 

6.  Zachary  Clayton  ~  Marketing 

| 

Sjfj 

7.  GROUP  DISCUSSION 

| 

5 

^  1 

ip  i!f 

L  m  ir,1 

f  ?  ideas  it  ccrpments  i  cat.  4  Q:Zi 

j|  Start  participants  in  this  activity  or  resource 

i^Startj  |  ^  |  y  \  Group  Systems  *  GSS 

tidfbsclt  Wi«S|6SS^k^j.»  .j  '  * 

>  Facilitator  says:  “At  this  point,  you  should  have  been  invited  to  join  an 
activity.  Please  click  on  Yes.”  (Note  to  Researcher:  In  some  cases  this  field 
will  be  blank.) 

>  Facilitator  says:  “You  may  receive  another  log-in  prompt.  Please  click  on 
OK.” 

>  Facilitator  says:  “You  should  now  see  a  list  of  six  names  and  a  category  called 
“Group  Discussion”.  Double  Click  on  Group  Discussion.” 

>  Facilitator  says:  “A  new  window  should  appear  on  your  screen.  This  is  a 
discussion  area  where  you  will  provide  comments  for  the  group  problem¬ 
solving  task.  At  this  point  your  cursor  should  be  in  the  large  field  at  the 
bottom  of  the  window.  This  is  the  box  where  you  enter  your  comments. 

Please  type  in  one  method  you  would  use  to  rank  order  the  list  of  names.” 

>  PAUSE 

>  Facilitator  says:  “Click  on  the  Submit  key  at  the  bottom  of  the  window  on  the 
left.  The  comment  you  entered  should  appear  in  the  notepad  above  the  large 
field.  Everyone  in  the  GSS  session  will  be  able  to  see  all  comments  submitted. 
Does  anyone  not  see  other’s  comments?” 
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>  For  a  Labeled  Session  the  facilitator  says:  “If  you  look  at  the  end  of  each 
comment  you  will  see  that  the  GSS  software  labels  the  person  who  entered  the 
comment.  You  should  see  our  choice  of  labels  (blue,  green,  red  or  yellow)  at 
the  end  of  each  comment.” 

>  Facilitator  says:  “As  your  group  brainstorms  and  you  enter  your  individual 
comments,  all  of  you  will  be  able  to  see  the  inputs  of  the  entire  group. 

Reading  others’  thoughts  and  ideas  allows  you  to  “piggyback”  off  each  other 
which  should  improve  your  group  brainstorming  process.” 

>  Facilitator  says:  “Now  that  we’ve  shown  you  how  to  enter  comments,  we  will 
now  have  you  perform  a  practice  session  before  we  move  into  the  problem¬ 
solving  task.  Your  group’s  task  is  to  discuss  possible  ways  your  group  could 
rank  order  the  names.  Any  and  all  comments  are  valuable,  including  ideas  on 
how  to  rank  the  names,  and  your  thoughts/opinions  of  each  other’s  ideas.  You 
will  have  a  couple  of  minutes  to  discuss  the  task  as  a  group.  At  the  end  of  the 
session  we  will  measure  group  consensus  on  how  you  ranked  the  list  by 
introducing  you  to  the  GSS  Vote  tool.” 

>  Facilitator  says:  “Please  begin  discussing  the  task.” 

>  Assistant  notifies  facilitator  when  time  reaches  5  minutes. 


>  Facilitator  says:  “Please  stop  discussing  the  task  at  this  time.” 

>  Facilitator  says:  “You’ve  had  plenty  of  time  to  discuss  possible  ways  to  rank 
order  the  list  of  names.  Now  it’s  time  to  actually  rank  the  names.  Hopefully, 
during  the  discussion  period,  your  group  decided  how  to  rank  the  list.  We  will 
now  introduce  you  to  the  GSS  Vote  tool  where  each  of  you  will  individually 
rank  the  list  of  names.  Please  close  the  Group  Discussion  window.” 


>  Facilitator  closes  training  categorizer  and  selects  voting  method  for  ballot  and 
clicks  OK. 


ti 


Select  Voting  Method 


-  Method 


r  f 


.5.'  W&  :  WW  WMM. 


El 


i  ! 


:  -♦  Rank  Order,  £  >  pue/False  ;  j 

f  CrNZkLa  ■  .  A- .  Pvnninr)  ............  <  ' 


ip . 


1 1  j  l  >  1  ^  Scale  '  |  |  Agree/Disagree  (5-point) 

. 

' '/  Ye$/No:  1  || jj  Custom  Method;'  J 


erf  Beleckf.'w v: 


mam 


I  WtM ' 


MHS 


•  'I'  ••V-vvY:----:- 

QK 


£ahcel  I 

. a . _ i 


Help  I 
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>  Facilitator:  Start  participants  in  Training  (Vote). 


j  GroupSy  stems  -  GSS  Study  -  Current  -  [Training  (Vote]J 

■IS  13 1| 

If*  £Se  Folders  Edit  Vote  Group  Options  Window  Help 

■  0  Agenda  •  people  ;  Whiteboard  ^Handouts  4  Opinion  ^Reports  ^Briefcase  1  og  ;  #4  Fir 

~  W$i tfc*  '  x| 

id  |  03  Folder  1st 

CO:'"  '■  '  -  ■ 

^  o  0  y  i  +: 

a,  ss  &  a 

ss  a  s  sis 

^  AFRL-  CIO  study  IMGT  699  Spring  2000 
^DemoAFRUCDO 
0  GSS  Labeled  Study-  clean  copy 

GSS  Unlabeled  Study  -  clean  copy 
0  VLSI  Collaboration  Demo  January  1 9, 2000 


1  Holly  Bower - 
2.  Susan  Peterson  -  Marketing 


3.  William  Elliott- Accounting 


4.  Albert  Smith  -  Marketing 


5.  Michelle  Zunga  -  Accounting 


6.  Zachary  Clayton  -  Marketing 


HI  1111  if  2:39PM 


>  Facilitator  says:  “You  have  been  invited  to  join  the  Voting  activity.  Please 
click  yes.  Now  you  should  see  the  original  list  of  names.  You  change  the 
sequence  of  the  list  by  clicking  and  dragging  an  item  to  the  position  in  the  list 
you  wish  to  move  it.  Please  begin  voting  by  re-ordering  the  list  now.” 


>  PAUSE 


>  Facilitator  says:  “Once  you  are  satisfied  with  your  list  order,  cast  your  ballot 
by  clicking  on  the  “cast  ballot”  icon,  which  is  the  2nd  from  the  left.  You  will 
receive  a  dialogue  box  asking  you  to  confirm  your  ballot.  Please  click  yes  and 
wait  for  further  instructions. 
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>  Ensure  voting  graph  tool  is  set  as  follows: 


Options 


>  Facilitator:  Open  the  result  window  (bargraph)  to  monitor  individual  votes. 


«  GroupSystems  -  GSS  Study  --  Current  -  [Moon  (Vote}] 

Is 


BlSBj 


0  First  Folder-  facilitator 

0  AFRL -  CIO  study  IMGT  699  Spring  2000  | 

0  Demo  AFRUCDO 
0  GSS  Labeled  Study-  clean  copy 

■  . :;  an  ^  ::ai 

0  GSS  Unlabeled  Study  -  clean  copy 
0  VLSI  Collaboration  Demo  January  1 9, 2000 


TV!- 


4.  Food  Concentrate 

■ 

5.  Life  Ran 

% 

i 

6.  Magnetic  Compass 

m: 

7.  One  Case  Dehydrated  Milk 

8.  Parachute  Silk  •; . 

if?*;: 

9.  Portable  Heating  Unit 

: 

10.  Signal  Flares  \ 

11.  Solar-Powered  FM  Receiver  Transmitter 

1 

12.  Stellar  Map  (of  the  Moon's  Constellation) 

13.  Two  .45-Caliber  Pistols 

Jsi 

-  . 

A  A  T\jun-l_nn.Prii  met  Tankc  nf  Hwnan - - - = — * 

>  When  n=number  of  participants,  the  group  is  done.  Display  the  results  with 


Hi 


HjVote  Spread  -  Training  (Vote] 


the  projector  and  explain  the  level  of  group  consensus 


Holly  Bower --  Accounting 


Group  consensus  (1.00  =  most  cons 


:•  3:14  PM 


GioupSystems  -  GSS'^|j|afi|gg 


Holly  Bower--  Accounting 

Oj  0 

JL 

0 

01 

Susan  Peterson  -  Marketing 

0  0 

if 

0 

. c 

William  Elliott- Accounting 

oi  o 

. i 

0 

Oj 

Albert  Smith  -  Marketing 

o|  o[ 

o' 

0 

oj 

Michelle  Zunga  -  Accounting 

’  o[  oi 

. of 

F 

oT  ~ 

Zachary  Clayton  -  Marketing 

"of ‘of 

“ol 

o“ 

of 

!  -J 

0 

S  X 

1  \ 

^  o 

1  -j 

0 

i  -| 

0 

>  Facilitator  says:  “During  the  actual  problem-solving  tasks  following  this 

training,  your  group  will  be  given  5  minutes  after  the  initial  vote  to  allow  your 
group  to  determine  if  everyone  is  satisfied  with  the  final  solution,  or  if  further 
discussion  is  needed.” 


4.  Training  exercise  complete. 

5.  Facilitator  says:  “I  will  now  be  closing  the  training  session  and  beginning  the  first 
exercise.  Please  do  not  enter  any  information  until  instructed.” 


6.  Facilitator  stops  participants  in  Training  (Categorizer). 

7.  Facilitator  stops  participants  in  Training  (Vote). 


8.  Facilitator  flips  DOWN  projector 
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GSS  Room:  Experiment  One 

1 .  Moon  Scenario  Script 

>  Facilitator  says:  “You  will  have  15  minutes  to  discuss  the  scenario  as  a  group. 
The  scenario  you  will  be  discussing  is  the  same  one  you  did  previously  as 
individuals.  At  the  end  of  the  15  minutes  you  will  each  rank  order  the  list 
individually.  The  ranking  results  will  be  consolidated  as  a  group  to  indicate  how 
well  the  group  reached  consensus.  At  this  time  the  group  will  have  another  5 
minutes  to  determine  if  you  have  reached  consensus.  If  the  group  did  not  reach 
consensus  you  should  try  to  resolve  any  differences.  The  group  will  then 
individually  rank  the  items  again.” 

>  Facilitator  says:  “During  the  15-minute  discussion  period,  focus  on  discussing 
each  item’s  merits,  not  on  where  each  item  should  be  ranked.  You  will  be  able  to 
rank  order  the  list  at  the  end  of  the  15  minutes.  It  is  important  to  focus  on 
discussion  not  on  how  to  rank  order  since  this  could  shut  down  conversation.  It  is 
normal  to  experience  a  lull  during  conversation,  but  this  is  part  of  group 
dynamics.” 

>  Facilitator  says:  “Researchers  will  not  answer  questions  dealing  with  the 
scenario  during  this  session,  but  will  assist  with  GSS  questions.  We  will  let  you 
know  when  there  are  5  and  2  minutes  left  in  the  session.” 

>  Facilitators  start  participants  in  Moon  Scenario  ~  Categorizer. 

>  Facilitator  says:  “At  this  point,  you  should  have  been  invited  to  join  an 
activity.  Please  click  on  Yes.”  (Note  to  Researcher:  In  some  cases  this  field  will 
be  blank.) 

>  Facilitator  says:  “Now  you  should  be  prompted  to  enter  an  author  tag.  Please 
click  on  OK.” 

>  Facilitator  says:  “You  have  a  list  of  15  items  and  a  category  called  “Group 
Discussion”.  Double  Click  on  Group  Discussion.” 

>  Facilitator  says:  “Are  there  any  questions?” 

>  Facilitator  says:  “You  may  begin  group  discussion  on  the  Moon  Scenario.” 


10:00 

FACILITATOR  INPUT:  YOU  HAVE  5  MINUTES  LEFT  TO  DISCUSS 
THE  SCENARIO  IF  NEEDED. 

13:00 

FACILITATOR  INPUT:  YOU  HAVE  2  MINUTES  LEFT  TO  DISCUSS 
THE  SCENARIO  IF  NEEDED. 
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>  Subjects  finish  initial  discussion. 

>  Facilitator  says:  “Please  close  your  Group  Discussion  Window.” 

^  Facilitator  says:  “We  will  now  open  a  voting  tool  for  your  use.” 

>  Facilitator  starts  Moon  (Vote). 

>  Facilitator  says:  “You  have  been  invited  to  join  the  Voting  activity.  Please 
click  yes.  Now  you  should  see  the  original  list  of  items.  Please  begin  voting 
by  re-ordering  the  list  now  just  as  you  did  in  the  training  session.” 

>  PAUSE  (1  minute) 

>  Facilitator  says:  “Once  you  are  satisfied  with  your  list  order,  cast  your  ballot 
by  clicking  on  the  “cast  ballot”  icon,  which  is  the  2nd  from  the  left.  You  will 
receive  a  dialogue  box  asking  you  to  confirm  your  ballot.  Please  click  yes  and 
wait  for  further  instructions.” 


>  Ensure  voting  graph  tool  is  set  as  follows: 


>  Facilitator:  Open  the  result  window  to  monitor  individual  votes.  When 
n=number  of  participants,  the  group  is  done.  Display  the  results  with  the 
projector  and  explain  the  level  of  group  consensus. 


:gBStaHp^:4Sf  f  [  I^GroupSyslemi  -  GSS  ...  Tgf  Micioiolt  Word  -  GSS_Lab..-.) 


>  Facilitator  says:  “You  have  the  next  five  minutes  to  discuss  the  results  further 
using  the  GSS.  If  you  are  satisfied  with  the  results  let  the  facilitator  know.  If 
not  you  will  be  given  the  chance  to  vote  again  at  the  end  of  the  five  minutes.” 

>  If  group  is  satisfied  with  original  outcome  then  go  to  STEP  2  else  do  the 
following: 

>  Facilitator  says:  “Go  ahead  and  vote  again.” 

>  Facilitator:  Open  the  result  window  to  monitor  individual  votes. 
Assistant  lets  Facilitator  know  when  the  group  is  done.  Display  the 
results  with  the  projector  and  explain  the  level  of  group  consensus. 

>  Facilitator  says:  “These  are  your  final  results.  “ 

2.  Facilitator  stops  Moon  (Vote). 

3.  Facilitator  flips  DOWN  projector 

4.  Facilitator  says:  “Feel  free  to  take  a  quick  5-minute  break  in  the  prep  room.  Please 
don’t  discuss  what  color  you  are.” 
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5.  Assistant  counts  number  of  comments  per  subject  and  creates  appropriate  feedback 
and  goal  charts. 

6.  Facilitator  stops  Subjects  in  Moon  Scenario  --  Categorizer 
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Prep  Room:  Feedback 

1 .  Subjects  come  back  from  break 

2.  Assistant  says:  “We  would  now  like  to  give  you  a  short  questionnaire  concerning 
your  groups  ranking  on  the  task  you  just  completed.” 

3.  Assistant  provides  subjects  with  5  item  commitment  to  ranking  Questionnaire 

4.  Goal  and  No  Feedback  or  Goal  Feedback  Script 

5.  Assistant  says:  “During  the  group  exercise  just  completed,  your  group  worked 
together  to  solve  a  problem.  Studies  have  shown  that  when  individual  members  of 
the  group  participate  fairly  equally,  the  meeting  will  produce  better  results.  For 
example,  as  you  can  see  in  the  graph  (show  graph  of  equal  proportion)  the 
participation  rates  were  almost  equal  among  the  group  participants.  The  next 
graph  shows  participation  rates  where  participants  did  not  participate  equally. 
What  problem  can  result  from  the  unequal  levels  of  participation?” 

6.  (Wait  for  group  to  respond. . .  Look  for  an  answer  such  as  subject  4  did  not 
participate  as  much  and  he  may  have  had  the  best  answer  while  subject  2 
dominated  the  meeting  with  his  ideas.  If  group  does  not  submit  the  answer  looked 
for,  provide  an  explanation. 

7.  Assistant  says:  “Studies  have  also  shown  that  the  more  comments  input  during  a 
meeting,  the  greater  the  chances  to  reach  a  high  quality  decision.  In  other  words, 
the  more  ideas  that  are  generated  the  better  the  chance  the  optimum  solution  will 
be  found  in  those  comments.” 

8.  Assistant  says:  GOAL  STATEMENT:  “In  the  next  task,  try  to  participate  equally 
while  maximizing  your  number  of  comments.” 

9.  IF  GIVING  FEEDBACK  READ  THE  NEXT  SECTION,  IF  NOT  STOP  AND 
PROCEED  TO  NEXT  TASK 


10.  Assistant  says:  “I  will  now  show  each  of  you  a  graph  showing  your  participation 
level  in  the  previous  task." 

11.  Assistant  provides  an  explanation  of  the  feedback. 

12.  Assistant  says:  “Next  to  the  bar  graph  of  each  individual  show  their  score.  Once 
all  subjects  have  looked  at  the  paper  give  them  the  paper  to  start  the  desert 
scenario.” 
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13.  Assistant  says:  “You  will  now  be  given  5  minutes  to  complete  the  desert  scenario. 
Please  follow  the  directions  on  the  page.” 

14.  Subjects  individually  complete  Desert  Scenario 

15.  Assistant  says:  “We  will  now  move  to  the  Task  room  to  continue  the  task. 
Remember  the  goal  to  participate  equally  while  maximizing  your  number  of 
comments.  Please  take  your  desert  scenario  and  questionnaire  with  you  and  place 
it  in  your  manila  folder.” 

16.  Researchers  move  subjects  to  Task  Room  (GSS  Room) 
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GSS  Room:  Experiment  Two 

1.  Desert  Scenario  Script 

>  Facilitator  says:  “You  will  have  15  minutes  to  discuss  the  scenario  as  a  group. 
The  scenario  you  will  be  discussing  is  the  same  one  you  did  previously  as 
individuals.  At  the  end  of  the  15  minutes  you  will  each  rank  order  the  list 
individually.  The  ranking  results  will  be  consolidated  as  a  group  to  indicate  how 
well  the  group  reached  consensus.  At  this  time  the  group  will  have  another  5 
minutes  to  determine  if  you  have  reached  consensus.  If  the  group  did  not  reach 
consensus  you  should  try  to  resolve  any  differences.  The  group  will  then 
individually  rank  the  items  again.” 

>  Facilitator  says:  “During  the  15-minute  discussion  period,  focus  on  discussing 
each  item’s  merits,  not  on  where  each  item  should  be  ranked.  You  will  be  able  to 
rank  order  the  list  at  the  end  of  the  15  minutes.  It  is  important  to  focus  on 
discussion  not  on  how  to  rank  order  since  this  could  shut  down  conversation.  It  is 
normal  to  experience  a  lull  during  conversation,  but  this  is  part  of  group 
dynamics.” 

>  Facilitator  says:  “Researchers  will  not  answer  questions  dealing  with  the 
scenario  during  this  session,  but  will  assist  with  GSS  questions.  We  will  let  you 
know  when  there  are  5  and  2  minutes  left  in  the  session.” 

>  Facilitators  start  participants  in  Desert  (Categorizer). 

>  Facilitator  says:  “At  this  point,  you  should  have  been  invited  to  join  an 
activity.  Please  click  on  Yes.”  (Note  to  Researcher:  In  some  cases  this  field  will 
be  blank.) 

>  Facilitator  says:  “Now  you  should  be  prompted  to  enter  an  author  tag.  Please 
click  on  OK.” 

>  Facilitator  says:  “You  have  a  list  of  15  items  and  a  category  called  “Group 
Discussion”.  Double  Click  on  Group  Discussion.” 

>  Facilitator  says:  “Are  there  any  questions?” 


>  Facilitator  says:  “You  may  begin  group  discussion  on  the  Desert  Scenario.” 


SfFIB 
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10:00 

FACILITATOR  INPUT:  YOU  HAVE  5  MINUTES  LEFT  TO  DISCUSS 
THE  SCENARIO  IF  NEEDED. 

13:00 

FACILITATOR  INPUT:  YOU  HAVE  2  MINUTES  LEFT  TO  DISCUSS 
THE  SCENARIO  IF  NEEDED. 
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>  Subjects  finish  initial  discussion. 

>  Facilitator  says:  “Please  close  your  Group  Discussion  Window.” 

>  Facilitator  says:  “We  will  now  open  a  voting  tool  for  your  use.” 

>  Facilitator  starts  Desert  (Vote). 

>  Facilitator  says:  “You  have  been  invited  to  join  the  Voting  activity.  Please 
click  yes.  Now  you  should  see  the  original  list  of  items.  Please  begin  voting 
by  re-ordering  the  list  now.” 

>  PAUSE  (1  minute) 

>  Facilitator  says:  “Once  you  are  satisfied  with  your  list  order,  cast  your  ballot 
by  clicking  on  the  “cast  ballot”  icon,  which  is  the  2nd  from  the  left.  You  will 
receive  a  dialogue  box  asking  you  to  confirm  your  ballot.  Please  click  yes  and 
wait  for  further  instructions.” 


>  Ensure  voting  graph  tool  is  set  as  follows: 


Options 


>  Facilitator:  Open  the  result  window  to  monitor  individual  votes.  When 
n=number  of  participants,  the  group  is  done.  Display  the  results  with  the 
projector  and  explain  the  level  of  group  consensus. 

>  Facilitator  flips  UP  projector 
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>  Facilitator  says:  “You  have  the  next  five  minutes  to  discuss  the  results  further 
using  the  GSS.  If  you  are  satisfied  with  the  results  let  the  facilitator  know.  If 
not  you  will  be  given  the  chance  to  vote  again  at  the  end  of  the  five  minutes.” 

>  If  group  is  satisfied  with  original  outcome  then  go  to  STEP  2  else  do  the 
following: 

>  Facilitator  says:  “Go  ahead  and  vote  again.” 

>  Facilitator:  Open  the  result  window  to  monitor  individual  votes. 
Assistant  lets  Facilitator  know  when  the  group  is  done.  Display  the 
results  with  the  projector  and  explain  the  level  of  group  consensus. 

>  Facilitator  says:  “These  are  your  final  results.  “ 

2.  Researcher  stops  Desert  (Vote). 

3.  Facilitator  flips  DOWN  projector 

4.  Facilitator  says:  “Lets  go  back  to  the  prep  room  to  finish  up.” 


105 


Prep  Room:  Wrap-Up 

1 .  Assistant  provides  subjects  with  Post-Session  Questionnaire 

2.  Subjects  complete  Post-Session  Questionnaire 

3.  Assistant  debriefs  subjects 

“The  experiment  you  just  participated  in  was  designed  to  measure  the  effect  of 
feedback  and  goal  setting  on  group  performance,  compare  different  levels  of 
anonymity  in  a  meeting  on  group  performance,  study  ideation  over  time,  and  evaluate 
the  influence  of  personality  types  on  groups.” 

“The  experiment  collected  data  on  the  quantity  of  comments  provided,  the  quality  of 
group  decision,  the  timing  of  ideas  generated,  and  participation  rates  from  various 
personality  groups.” 

“I  would  like  to  thank  you  for  your  participation  in  this  experiment.  Do  you  have  any 
other  questions  about  the  experiment  you  participated  in  today  or  on  Group  Support 
Systems?” 

[Pause  for  questions.] 

“Please,  if  you  know  others  who  are  likely  to  participate  in  this  experiment,  please  keep 
the  details  of  the  experiment  to  yourself  in  order  to  avoid  biasing  our  final  results  and 
jeopardizing  the  continuation  of  this  study.” 

4.  Researchers  collect  all  handouts,  data,  disks,  etc.  and  ensures  all  are  labeled 
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Appendix  F:  Consent  Form 


Study  Overview 

Welcome  to  the  experiment.  The  following  is  a  general  description  of  the  study  and  a 
reminder  of  your  rights  as  a  potential  subject.  As  in  any  study,  your  participation  is 
completely  voluntary.  If  now,  or  at  any  point  during  the  study,  you  decide  that  you  do 
not  want  to  continue  participating,  please  let  the  experimenter  know  and  you  will  be 
dismissed  without  penalty.  Also,  please  remember  that  your  name  will  not  be  associated 
with  any  of  the  information  that  you  provide  during  the  study.  All  of  the  information  you 
provide  is  absolutely  anonymous  and  confidential. 

In  this  study,  you  will  be  working  as  part  of  a  group  to  complete  two  group  tasks. 
You  will  also  be  asked  to  complete  two  questionnaires  during  the  study.  You  will  first  be 
given  a  questionnaire  to  complete,  then  you  will  complete  the  first  task  as  a  group,  after  a 
short  break  you  will  be  given  the  second  task  to  complete  as  a  group,  and  finally,  you  will 
be  given  a  second  questionnaire  to  complete.  The  experimenter  will  give  you  more 
specific  instructions  later  in  the  study.  If  you  have  any  questions  or  concerns  at  this  time, 
please  inform  the  experimenter. 

For  further  information 

The  Air  Force  Institute  of  Technology  faculty  members  responsible  for 
conducting  this  research  are  Maj.  Michael  Morris  and  Maj.  Paul  Thurston.  They  would 
be  happy  to  address  any  of  your  questions  or  concerns  regarding  this  study.  Maj.  Morris 
can  be  reached  at  255-3636  ext  4578  and  Mai.  Thurston  can  be  reached  at  255-6565  ext 
4315. 

If  you  would  like  to  participate  in  this  study,  please  sign  in  the  space  provided.  Your 
signature  indicates  that  you  are  aware  of  each  of  the  following:  1)  the  general  procedure 
to  be  used  in  this  study,  2)  your  right  to  discontinue  participation  at  any  time,  and  3)  you 
and  your  name  will  not  be  associated  with  any  of  the  information  you  provide. 


Printed  Name: 


Signature: 


Date:. 


107 


Appendix  G:  List  of  Names 


Holly  Bower  -  Accounting 
Zachary  Clayton  -  Marketing 
William  Elliot  -  Accounting 
Albert  Smith  -  Marketing 
Susan  Peterson  -  Marketing 
Michelle  Zunga  -  Accounting 
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